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PREFACE 


This Note describes the findings of the Automation and Robotics panel, one of eight 
project panels established by RAND to evaluate submissions to the Space Exploration 
Initiative (SEI) Outreach Program, also called Project Outreach. Project Outreach is a 
NASA-sponsored program to elicit innovative ideas, concepts, and technologies for space 
exploration. The project was sponsored by Project AIR FORCE and by RAND’s Domestic 
Research Division, with technical oversight provided by the Assistant Secretary of the Air 
Force (Space). 

The findings of other RAND panels are reported in the publications listed below. 

Space and Surface Power for the Space Exploration Initiative: Results from Project Outreach , 
C. Shipbaugh, K. Solomon, M. Juncosa, with D. Gonzales, T. Bauer, and R. Salter, N-3280- 
AF/NASA, 1991. 

Space Transportation Systems, Launch Systems, and Propulsion for the Space Exploration 
Initiative: Results from Project Outreach, T. Garber, J. Hiland, D. Orletsky, B. Augenstein, 
and M. Miller, N-3283-AF/NASA, 1991. 

Human Support Issues and Systems for the Space Exploration Initiative: Results from Project 
Outreach , J. Aroesty, R. Zimmerman, and J. Logan, N-3287-AF/NASA, 1991. 
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SUMMARY 


BACKGROUND 

President Bush stated his objectives for a Space Exploration Initiative (SEI) on July 
20, 1989. He called for a program that includes establishing a permanent outpost on the 
Moon and sending a manned mission to Mars. In response to the President's announcement, 
NASA conducted a 90-day study that presented a variety of strategies for accomplishing 
those objectives. 

Subsequently, Vice President Quayle, Chairman of The National Space Council, asked 
NASA to take the lead in identifying new and innovative approaches for traveling to the 
Moon and Mars and for living and working on both. Accordingly, NASA solicited ideas 
through the SEI Outreach Program, which had three principal components: 

1. Direct solicitation of ideas from academic institutions, private enterprise and the 
general public. 

2. Reviews of federally sponsored research. 

3. A study by the American Institute of Aeronautics and Astronautics (AIAA). 

NASA asked RAND to evaluate the results of the direct solicitation effort and provide 
that evaluation to the Synthesis Group chaired by Thomas P. Stafford, Lieutenant General, 
USAF (ret.). The results from the review of federally sponsored research and the AIAA study 
will also be available to the Synthesis Group. The Synthesis Group will make a further 
evaluation and synthesize at least two distinctively different SEI architectures and will 
submit its recommendations to NASA and The National Space Council. 

A total of 52 submissions were received in the Automation and Robotics (A&R) area 
during Project Outreach. About half of the submissions (24) contained concepts that were 
judged to have high utility for the Space Exploration Initiative (SEI) and were analyzed 
further by the robotics panel. These 24 submissions are discussed and analyzed in this Note. 

Three types of robots were proposed in the high-scoring submissions: structured-task 
robots (STRs), teleoperated robots (TORs), and surface exploration robots. Several advanced 
TOR control interface technologies were proposed in the submissions. Many A&R concepts or 
potential standards were presented or alluded to by the submitters, but few specific 
technologies or systems were suggested. 
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RECOMMENDATIONS 

Review of the submissions and further research in A&R issues has led the Project 
Outreach A&R panel to make the following observations and to submit the following 
recommendations for consideration by the Synthesis Group: 

• Systematically integrate SEI robots, work environments, and systems. 

• Develop structured-task robots for SEI. 

• Adapt and develop advanced TOR control interfaces that enable telepresence. 

• Evaluate the architectural implications of using TOR telepresence control in SEI. 

• Reevaluate and harmonize early SEI remote sensing data collection requirements 
with later SEI robotic mission requirements. 

• Conduct tradeoff studies to select optimum mobility and navigational subsystems 
for SEI surface exploration robots. Teams of complementary exploration robots 
should be considered in these tradeoff analyses. 

• Conduct tradeoff studies to determine the most cost-effective and productive 
development path towards autonomous robots. 

• Review NASA's evaluations of A&R effort for Space Station Freedom. 

Below we discuss these recommendations in more detail. 

Integrate SEI Robots, Work Environments, and Systems 

Most human work environments can be unstructured because humans can easily and 
rapidly acjjust to unanticipated changes or events in their environment. Such human 
adaptability and flexibility result from our sophisticated sensing, planning, navigation, and 
movement skills. The current state of the art in robotics cannot provide systems that 
faithfully mimic these human capabilities; thus, SEI work environments in space and on the 
surface of the Moon or Mars must be carefully designed with the current limits of robotics in 
mind. SEI robot end-effectors should all be designed and manufactured to a limited set of 
end-effector design rules, so different robots can use the same end-effectors for several 
manipulation tasks. And SEI components should be designed in a complementary fashion so 
they can be manipulated efficiently by robots using such standardized end-effectors. 

A critical area being ignored in the United States, but under consideration in Japan, is 
the development of space facilities that make extensive use of robots in their normal 
sequence of assembly, maintenance, and repair. Robots are still viewed in the United States 
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as gadgets or tools that are added to a structure to be constructed and maintained primarily 
by people. Extensive design exploration and demonstration efforts should be initiated to 
provide the United States with options for space and planetary systems that are primarily 
constructed, maintained, and repaired by robots. This theme was mentioned only 
tangentially in the Outreach submissions but has emerged as a critical recommendation from 
the A&R panel’s own analysis. 

Perhaps the most important issue involved in systemically integrating SEI robots, 
work environments, and systems is capturing and maintaining configuration control over 
SEI system designs. Detailed engineering design data should be captured in a common 
digital format and made portable so that it can be used by different system contractors 
during design and manufacturing and by robots in space during assembly and repair 
operations. Automated capture of SEI systems designs has been made possible with the 
advent of integrated Computer Aided Design/Computer Aided Manufacturing (CAD/CAM) 
tools. Transportability of CAD/CAM files is also being improved with the introduction of 
commercial standards such as the emerging Electronic Data Interchange Format (EDIF) 
CAD/CAM standard [Ref. 1]. NASA should monitor the development and use of CAD/CAM 
tools and standards in the semiconductor and other industries and adapt these increasingly 
powerful design tools to SEI systems and robots. 

Develop Structured Task Robots for SEI 

The most productive robots on Earth are STRs. They have transformed the Japanese 
auto and semiconductor industries. Now the Japanese install as many robots every year as 
exist in the entire U.S. industrial base [Refs. 2, 3]. Even more productive robots will be 
needed for SEI if the President’s ambitious mission goals are to be met within the specified 
time frame and within future budget constraints. 

Much further research into the use of STRs in space is required. The work 
recommended in submission #100378 should be greatly expanded for SEI. Assembly tasks 
should be made easy and modular, enabling STRs to be used wherever feasible at 
extraterrestrial operations nodes. 

Review of the submissions and this panel’s research and inquiries indicate that NASA 
A&R research and development activities may be too tightly focused on expensive one-of-a- 
kind high-technology developments like the Flight Telerobotic Servicer (FTS). 1 It is 
nevertheless unfortunate that the FTS program has recently been cancelled. While the FTS 

1 The FTS program has recently been downgraded from a full development program to a 
technology demonstration project. 
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program is a necessary and ambitious technology demonstration project, SEI funds should 
also be allocated towards development of STR work environments and STRs for specific SEI 
applications. These activities can help revive the moribund U.S. commercial robotics 
industry and will also provide a natural “upstream” technology base for the eventual 
colonization and industrialization of the Moon. 

Adapt and Develop Advanced TOR Control Interfaces 

Submissions *100695, #100338, #101469, #100827, #100336, #101317, and others 
propose thatTORs be used for many SEI assembly, processing, repair, and exploration tasks. 
Because TORs can be remotely controlled by humans, they can operate in unstructured 
environments and are more flexible and adaptable than STRs. They also require much less 
complex real-time software than autonomous robots. As a consequence of this, a variety of 
TORs have been developed for commercial and space applications, while autonomous robots 
have yet to be realized. However, most TORs available today are cumbersome to operate and 
typically perform manipulation tasks much slower than humans. For example, it is 
estimated that the FTS in its initial configuration will perform manipulation tasks in space 
at a significantly slower rate than a well-trained astronaut in an extra vehicular activity 
(EVA) spacesuit. The performance limitations of current TORs have therefore prompted 
researchers to develop new TOR control interfaces to improve TOR productivity. 

NASA researchers were among the first to develop new and innovative display and 
interactive computer control technologies, such as “eye phones" and “power gloves,” which 
offer tremendous promise as TOR control interfaces. Now commercial companies, both in the 
United States and Japan, are racing to refine and extend these technologies for many 
different consumer, scientific, and business products. In addition, HDTV, high-resolution 
flat-panel displays, and new three-dimensional display volume systems are being developed. 
The leading edge of development for these technologies is now being pushed faster and 
harder in the commercial world. NASA needs to keep abreast of these new developments, 
test new systems for TOR control, and integrate those that demonstrate their worth into 
future TOR systems, such as the FTS. These new technologies will allow NASA astronauts 
and the general public alike to experience SEI missions first-hand through telepresence.^ 

^Telepresence can be briefly defined as the creation for the individual user of a realistic, 
detailed, and complete artificial senaorium which “tricks* the user into believing he or she is present at 
a remote location. Computer and TOR control interfaces which exhibit telepresence have been called 
virtual environments, artificial realities, or cyberspaces by researchers, futurists, and science fiction 
writers. 
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New commercial speech synthesis and recognition products also are poised to enter the 
marketplace. NASA should monitor these developments so these capabilities can be quickly 
and cost-effectively integrated into new TOR control interfaces. 

Emerging TOR control technologies and advances in computer simulation may also 
permit development of radically new control interfaces that promise great increase in TOR 
operator productivity and the effective radius of TOR control from thousands to millions of 
kilometers. Many Project Outreach submissions suggested development of advanced TOR 
control interfaces capable of telepresence. One submission in particular (#100317) described 
in broad conceptual terms the enormous potential benefits of using these new technologies 
for TOR control. 

NASA needs to study these emerging technologies to see how they can best be used to 
control TORs and to see if they lead to new strategies for obtaining higher forms of machine 
autonomy. 

Evaluate the Architectural Implications of TOR Telepresence Control 

TORs may be used extensively in many phases of SEI operations. A significant 
amount of TOR coordination, mission planning, and real-time retasking will be required, 
especially for complex and TOR-intensive operations like assembly of Mars Transfer Vehicles 
(MTVs) or lunar base construction. If telepresence technology is used for TOR control, even 
more coordination may be necessary because TOR operators will be sensorially centered at 
the remote site where their TORs operate and not at their control stations. 

By making analogies to certain military operations and practices, it is conjectured that 
TOR Command, Control, and Communications (C3) centers will be required to efficiently and 
safely perform TOR supervision, coordination, and task planning. Depending upon the 
sophistication of TOR control available in the time frame of SEI, TOR C3 centers may be 
required at each major extraterrestrial SEI operations center. Although different 
terminology is used by the author, submission #100337 suggests development of such TOR 
C3 centers. 

TOR command and control manpower, power, habitat, and communications 
requirements must be studied by NASA and included in future SEI architecture studies. The 
most significant implication of the widespread use of TORs and the incorporation of 
telepresence control in SEI is the greatly increased communications burden SEI space 
networks may have to support. If one conjectures that HDTV-like display devices are used 
for stereoscopic control of each TOR, then approximately two HDTV channels will have to be 
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supplied for every TOR controlled from a distant location. 3 New developments in image 
compression and distributed simulation technologies will be required to reduce TOR 
command and control communications requirements and make SEI TOR telepresence control 
a reality. NASA should carefully monitor developments in these areas. 4 

Deepen SEI Robot Mission Planning 

As the SEI program proceeds over the next quarter-century, SEI operations will 
increase in scope and complexity. Succeeding generations of SEI robots will depend upon 
and exploit data collected from previous SEI and NASA missions. Data collection 
requirements on early missions should therefore be carefully determined with later SEI 
mission needs in mind. Synergies may exist between early SEI data collection efforts and 
later exploratory, construction, or resource extraction missions. If high-resolution data are 
collected on early exploratory missions, they may prove useful for many purposes and could 
reduce the cost and complexity of follow-on robotic systems, such as lunar rovers or base- 
construction robots. For example, as pointed out in submission #101067, the size and cost of 
lunar rovers could be reduced if data collected in early lunar remote sensing missions could 
be used to determine lunar rover “road networks” free of obstacles larger than 0.1 m. In 
addition, such data collection efforts would provide scientists and prospectors with an 
unprecedented geologic record of the lunar and perhaps Martian surfaces. 

High resolution imaging (0.1 m) of the Moon is feasible and could be carried out at a 
number of wavelengths. NASA should examine innovations in new sensor technologies and 
small satellite developments (Lightsats) to see if Lunar Observer or Martian Observer 
spacecraft should be augmented by new lightweight remote sensing systems that could not 
only provide higher resolution optical imagery but could also image permanently dark craters 
near the lunar poles [Ref. 5]. 

3 The FTS vision subsystem is composed of four ordinary (NTSC) video cameras: two 
anthropomorphically positioned on the robot’s “head” and the other two placed at the wrist of each of 
the FTS’s two robot arms [Ref. 4]. Some SEI TORs may also require more than two video channels even 
if high resolution imaging systems, such as HDTV, are used. Further TOR vision and control research 
is needed to answer these questions. 

4 The many other technologies beside high resolution image displays required for development of 
TOR telepresence control are described in detail in the body of this Note. However, the requirement to 
transmit high resolution imagery from the robot to the TOR controller places the greatest burden on 
the intermediate communications network. High resolution imagery is an essential component of 
telepresence, as it helps to embed die TOR controller in a realistic artificial sensorium. 
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Surface Exploration Robots 

A number of surface exploration robots have been proposed in various Project 
Outreach submissions to perform exploration tasks over various types of terrain 
(submissions #100336, #101067, #100815, #100337, #101325, #100339, and #100343). They 
can be grouped according to the mobility and navigation concepts they employ. Tradeoff 
studies need to be conducted comparing various mobility and navigation concepts to select 
those that could best fulfill SEI mission objectives. Submission #100343 proposes that robot 
teams be used to explore the Martian and lunar surface. Such a team may offer more terrain 
flexibility and may be more cost-effective than employing a small number of identically 
configured multipurpose rovers. 

Transition to Autonomous Robots 

One key SEI robotics programmatic issue over the next twenty years will be the 
schedule development risk for semi-autonomous or autonomous robots. Versatile 
autonomous robots capable of operating in unstructured SEI environments (an unprepared 
planetary surface, or free-flying LEO) will require many sophisticated capabilities. These 
capabilities require development of large, error-free, software codes and, as with all software 
development, the risk must be considered high. Initial operating capability (IOC) dates for 
autonomous robots cannot be predicted and may not be achievable without an enormous 
investment in software development infrastructure. 

Long-term tradeoff studies need to be performed by NASA and updated annually or 
biannually to determine the most cost-effective and technically feasible long-term 
autonomous robot development plan and to determine the balance between TOR and 
autonomous robot research and development. In addition, such assessments could also be 
used to determine which key subsystem technologies must be targeted for further 
development. If current technology trends continue, TORs equipped with telepresence 
control interfaces and limited forms of autonomy will prove to be the preferred development 
path. 

Several submissions (#100342, #100345, #100348, #100442, and #100333) recommend 
that in order to develop autonomous robots, NASA should adapt or develop emerging 
artificial intelligence technologies, autonomous navigation software, and new modular robot 
control and software standards such as the NASA/NBS Standard Reference Model 
(NASREM) and the USAF Next Generation Controller Project for a Standard Open System 
Architecture Specification (SOSAS). While these standards are rather general in nature at 
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this time, NASA can certainly profit from examination of these systems. With regard to 
development of advanced software products such as expert systems, a careful examination by 
NASA of the associated software development risks will be needed. 

Review NASA’s Evaluation of A&R Effort for Space Station Freedom 

The United States space program would be impossible without a level of automation 
and robotics that reflects, to some extent, the general state of the art. However, over the past 
twenty years, the dominant role of military and NASA agencies in A&R research and 
development has been sharply reduced while the role of commercial industry has increased 
proportionally. A major challenge to NASA is simply maintaining an awareness of A&R 
advances and how these technologies are being used in new ways in the commercial world 
(use of CAD/CAM tools in the semiconductor industry is one example). Implementation of 
evolving A&R technologies is an enormous challenge to the agency. At the direction of 
Congress, NASA has conducted a continual review of the implementation of A&R within the 
Space Station Freedom. A&R implementation efforts have been reviewed approximately 
every six months since 1985 [Refs. 6-15]. We recommend that the Synthesis committee 
review NASA’s evaluations of the Space Station Freedom effort to see how advanced A&R 
could be incorporated into SEI. Such a review will reveal the many difficulties, both human 
and technological, that lie ahead and, at the same time, the great motivations for pressing 
ahead. 
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I. INTRODUCTION 


This Note describes the results of RAND’s management of the direct solicitation 
component of the Space Exploration Initiative (SEI) Outreach Program, a program designed 
to solicit creative ideas from academia, research institutions, private enterprise, and the 
general public to help in defining promising technical areas and program paths for more 
detailed study. In addition to managing and evaluating the responses or submissions to this 
public outreach program, RAND conducted its own analysis and evaluation relevant to SEI 
mission concepts, systems, and technologies. The screening and analysis of Outreach 
submissions was conducted between July and October 1990, and involved staff and 
consultants throughout RAND’s departments and research divisions. 

Eight panels were created to screen and analyze the submissions. These panels 
encompassed: 

• Space and Surface Power 

• Space Transportation Systems, Launch Systems, and Propulsion 

• Automation and Robotics 

• Human Support 

• Structures, Materials, Mechanical Systems, and In-Situ Processing 

• Communications 

• Information Systems 

• Architectures and Missions 

This Introduction describes the overall methodology used in submission handling and 
analysis, as well as some general results and observations. The body of the Note contains the 
analyses and evaluations of the Automation and Robotics panel. 

BACKGROUND 

President Bush has called for a Space Exploration Initiative that includes establishing 
a permanent base on the Moon and sending a manned mission to Mars. The national space 
policy goals developed by the National Space Council and approved by President Bush on 
November 2, 1989, were the following: 

• Strengthen the security of the United States; 

• Obtain scientific, technological, and economic benefits; 
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• Encourage private sector investment; 

• Promote international cooperative activities; 

• Maintain freedom of space for all activities; 

• Expand human presence and activity beyond Earth orbit into the solar system. 

To support these goals, Vice President Quayle, Chairman of the National Space 
Council, has asked NASA to take the lead in identifying new and innovative approaches that 
will be required to travel to the Moon and Mars and to live and work productively on both 
worlds. Accordingly, NASA has begun to solicit new ideas and concepts for space exploration 
that will define promising mission paths for detailed study. The SEI Outreach Program has 
three principal components: 

1. Direct solicitation of ideas from academia, nonprofit organizations, for-profit 
firms, and the general public. 

2. Reviews of federally sponsored research. 

3. A study by the American Institute of Aeronautics and Astronautics (AIAA). 

The results of the three efforts listed above will be presented to a Synthesis Group 
chaired by Thomas P. Stafford, Lieutenant General USAF (ret.). The Synthesis Group 
received a number of ideas from various sources, collected additional information, and 
conducted detailed analysis. This process resulted in a synthesis of ideas. The 
recommendations of the Synthesis Group will be presented to the NASA Administrator and 
the National Space Council. From this process, a number of alternative mission paths will 
emerge for detailed study over the next few years. In addition, the process is expected to 
yield innovative technologies and system concepts for possible development. 

GENERAL OBSERVATIONS ON THE SUBMISSIONS 

Our first observation is that the submissions did not contain any new scientific laws, 
principles, or wholly new areas of technology. For example, some submissions suggested 
applications of high-temperature superconductivity, which five years ago could have been 
considered a “new* technology. However, superconductivity was first discovered in the early 
1900s, and the possibility of high-temperature superconductors was discussed soon 
afterward, so it should be understood that “new” technology areas are a matter of 
perspective. 

However, the submissions did contain a number of old ideas that have new 
implications in the context of the SEI. For example, several submissions included the 
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concept of a spacecraft hovering at a libration point, a concept that has been proven by 
NASA’s International Sun-Earth Explorer-3, which was put into orbit around the Sun-Earth 
libration point, L-l, in 1978. Libration concepts take on considerably new meaning in the 
context of potential use as transportation nodes for a Mars mission. See R-4112-AF/NASA 
for further discussion. 

The submissions also contained ideas that had not been heretofore supported by the 
submitter’s organization, which may have been an industrial firm, university, or NASA itself. 
This is a natural consequence of the priority planning process and resource allocation 
decisions of each individual organization. Thus, many of the submitted ideas are not 
completely new, but simply have not received much support heretofore. 

Lastly, we observe that the submissions were sufficiently diverse to support a wide 
range of SEI mission concepts and architectures. 

THE SUBMISSION PROCESS 

Figure 1 presents a flow diagram of the Outreach evaluation process. RAND mailed 
out 10,783 submission packets, in addition to the 34,500 that were mailed out by NASA. A 
total of 1697 submissions were received and were initially processed by a subcontractor firm, 
KPMG Peat Marwick. Of the 1697 submissions received, 1548 were judged by Peat Marwick 
to contain sufficient information for screening by RAND. The screening process selected 
approximately 140 submissions for more formal analysis. The output of that analysis process 
is a set of priority submissions and recommendations reported in this and several companion 
Notes. 

For further discussion of the sources of submissions and their management by RAND, 
please see App. A. 

THE SCREENING PROCESS 

The screening process objectives were to: 

• Assure relative insensitivity to the quantity of submissions; 

• Select submissions to be analyzed at length; 

• Review each submission by at least two technical experts working independently; 

• Examine robustness by providing more than one ranking method; 

• Maintain analytic rigor. 
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Fig. 1.1 — RAND’* outreach process 

The first objective of the screening process was to assure a good capability to deal with 
the quantity of submissions, whatever their numbers. Therefore, we constructed a 
“production line” for processing that would enable insensitivity to the quantity of 
submissions. 

The next task of the screening process was to decide which submissions would be 
analyzed. We decided that the range and depth of our analysis would have to be a function of 
(1) the resources available, (2) the perceived quality of submissions across panels, and (3) the 
relative importance of topics to the overall SEI program. One obvious pair of important 
panels (because of the tradeoffs between them) was the Human Support panel and 
Transportation panel. 

In the screening process, each submission was reviewed by at least two technical 
experts working independently. We allowed for robustness by providing more than one 
ranking method. A related goal was to maintain analytic rigor through the maintenance of 
tracking systems to enable later analysis of our methodology. 
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“Multi-attribute derision theory” was used in the screening process; that is, a group of 
attributes was used to evaluate each submission. The panels chose to score their various 
submissions using the same five principal attributes: 

• utility 

• feasibility 

• safety 

• innovativeness 

• relative cost 

Each panel tailored its own criteria for scoring an attribute according to the panel’s 
specific needs. For example, “safety” meant a very different thing to the Transportation 
panel than it did to the Communications panel. 

Attributes were independently scored by two or more reviewers on a scale of one to 
five, with five being the best. Written justification for the scoring was input into the text 
field in the database. We used a widely accepted Macintosh relational database, Fourth 
Dimension by ACIUS, Inc., for storing and using the various information components of each 
submission. 

If any attribute score varied by more than one among different reviews of the same 
submission, the submission was reviewed again, this time with the panel chairman 
participating with each of the original reviewers. However, there was no pressure to reach 
consensus. 

A complete discussion of the quantitative means by which panels used their attribute 
criteria to rank and evaluate submissions is provided in App. A. The specific criteria used by 
the Automation and Robotics panel in assigning attribute scores are also discussed in App. A. 

THE ANALYSIS PROCESS 

The object of the analysis process was to select the submissions to be recommended for 
further consideration by the Outreach Synthesis group. Where possible, we analyzed the 
submissions quantitatively within the context of the important performance tradeoffs in their 
respective technical areas. 

Each panel prepared a draft paper on the results of its analysis in its area of technical 
responsibility. Each draft paper is organized into technical discussions of the important 
technical sub-areas identified by that panel. Where possible, important performance 
tradeoffs in each sub-area are examined quantitatively. 
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Submissions that arrived with no backup paper — no detailed substantiating 
information or documentation — were analyzed in the context of the technical discussions of 
the appropriate sub-areas, thus providing necessary background. The majority of 
submissions did not, in fact, include backup papers, making an extended analytical 
discussion almost mandatory in most cases. 

SCOPE OF THE AUTOMATION AND ROBOTICS PANEL 

Project Outreach submissions that explicitly proposed the use or development of 
robots, automated systems, or robot control systems to accomplish SEI mission objectives 
were evaluated by the Automation and Robotics (A&R) panel. Submissions that proposed the 
use or development of specific technologies such as robotic or automated subsystems 
(Artificial Intelligence (AI), for example) were also evaluated by the A&R panel. Because 
A&R subsystems were evaluated, some overlap exists between the scope of the A&R panel 
and other panels of Project Outreach (for example, the Information Processing panel). Some 
overlap with the other panels is inevitable in the A&R area because robotics is a 
multidisciplinary field in which computer hardware, sensors, controllers, motors, displays, 
and advanced software products all play a key role. 

STRUCTURE OF THE NOTE 

Section II provides background on potential SEI robotics tasks and presents a robot 
classification scheme. The essential characteristics of robot work environments are also 
described. 

Section III contains our discussion of the submissions. Submissions are grouped into 
several broad technical categories and themes to enable a coherent comparative discussion. 

In each category, a theme or set of themes is elaborated, to place each of the submissions in a 
common context. Section IV presents our conclusions. App. A describes the specific criteria 
we used in scoring submissions; App. B provides a list of all submissions reviewed by the 
Automation and Robotics panel. 



- 7 - 


II. CLASSIFICATION OF ROBOTIC TASKS AND ROBOTS 


The use of manpower is extremely constrained in earth orbit and beyond, and will 
likely remain so for the foreseeable future. It is inefficient and potentially dangerous for an 
astronaut to work outside the cabin of a spacecraft or station. Inefficiencies arise from the 
restrictions imposed by the space suit, the necessity for lengthy preparation and desuiting, 
suit maintenance and repair, use of astronaut pairs (the buddy system), and the requirement 
for an on-board astronaut to continuously monitor the pair working outside. 

Cosmonaut extra vehicular activity (EVA) experience has led the Soviets to start 
developing robots for construction and repair in space. During a recent MIR mission in 
which cosmonauts performed EVA, an air-lock hatch proved balky, and the cosmonauts 
almost lost their lives. Soviet experience has been that after three to four hours of EVA the 
cosmonaut is exhausted and cannot do useful work for a significant period of time 
afterwards. 1 

Even inside a space facility, manpower is in short supply due to small crew size, the 
need for sleep, and the pursuit of other duties. In addition, in low earth orbit (LEO) or on the 
lunar surface, an astronaut working outside may be exposed to hazardous high-speed debris, 
cosmic rays, and solar radiation. It would be advantageous for any task that must be 
performed outside inflight spacecraft, or at lunar and Martian bases, to be performed by 
robots rather than humans. The types of tasks that robots can perform will steadily increase 
as robotics technologies advance and the work situations are designed to accommodate 
robots. In this section, we discuss potential SEI robotic tasks and develop a robot 
classification scheme and definitions of structured and unstructured work environments. 

POTENTIAL SEI ROBOTIC TASKS 

Potential SEI robotic tasks fall into two main areas: (1) operations in space, and 
(2) operations on the lunar and Martian surfaces. We discuss each below. 

Operations In Space 

The first major construction task attempted by humans in space will be the assembly 
of Space Station Freedom (SSF). SSF construction is scheduled to take place over many 
years in the latter half of this decade. In its earliest design phases, SSF was to be assembled 

1 Meeting of RAND Project Outreach panel leaders with Victor M. Surikov, Deputy Director, 
Central Research Institute of General Machinery, U.S.S.R., November 6, 1990. 
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completely by EVA astronauts. However, as the complexity of the assembly operation 
became more apparent, it became clear that too many astronaut hours would be required to 
perform this task with humans alone. The United States Congress mandated development of 
a robot, the Flight Telerobotic Servicer (FTS), as part of the Space Station Freedom program 
[Refs. 1, 16, 17]. This mandate was motivated by the desire to accelerate the technology for 
future industrial benefits. The FTS would have fortuitously enabled NASA to reduce the 
EVA demand for SSF assembly if this robot had not been removed from the SSF program 
[Ref. 43]. The billion-dollar program is now well under way. In addition, both Germany and 
Japan are developing FTS-like robots for use on SSF. 

Many SSF assembly tasks could be performed by robots. Robots can assemble truss 
structures and secure habitats to other modules and to the central keel trusses of the space 
station. Solar arrays and other systems which can only be assembled or connected to other 
systems in space could also be handled by robots. 

Robots equipped with video cameras or other sensors could also monitor the exterior 
and interior of SSF once assembly is complete. Video taken by EVA robots could be fed 
directly into habitat or laboratory modules of the station or via communications relays to 
NASA ground stations where people can safely monitor the status of SSF. 

More advanced robots may be able to independently monitor the status of certain SSF 
subsystems. If a defective subsystem or module were detected during routine monitoring, 
the robot could then advise personnel in SSF or on Earth. Again, depending upon the 
capabilities of the robot, it could independently cany out repair or replacement operations. 

In many of the reference architectures described in the NASA 90 Day Study [Ref. 18], 
the SSF plays a key role as an assembly and transportation hub; thus the robotic tasks 
described above could also be important components of SEI operations in space. In addition, 
LTVs, MTVs, and their cargo would be assembled, integrated, and tested at SSF or other 
space facilities. While LTVs may be brought to LEO in one piece, or may travel directly to 
low lunar orbit, robots may be required to service LTVs found to have problems after launch 
or after returning from the Moon. 

Robot space probes will continue to be used in the exploration of space, and several 
remote sensing satellite systems are part of the reference architectures described in the 
NASA 90 Day Study. The Lunar Observer system will image and map large parts of the 
lunar surface and will help to enlarge the database on the Moon’s geology, resources, and its 
historical part in the evolution of our solar system. The Mars Observer robot spacecraft will 
perform a similar mission. As SEI architectures are refined and further developed, more 
highly capable robot probes may be employed to characterize the surface and atmospheres of 
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both planets. These robot spacecraft will be capable of highly autonomous activity within 
their mission profiles and will also be capable of being redirected from Earth. 

The most demanding space assembly and repair tasks may well be the assembly and 
servicing of Mars Transfer Vehicles (MTVs). MTVs may have to be huge vehicles. MTV 
habitats may have to be covered with enormous shields to protect the human crew from 
galactic and solar radiation. Large-volume chemical fuel tanks may have to be integrated 
with the spacecraft in space. Alternatively, large nuclear power plants may have to be 
integrated and fueled in high altitude, nuclear-safe orbits above the Earth. Finally, if 
effective and safe countermeasures to the biological effects of microgravity cannot be found, 
the habitat portions of MTVs will have to rotate to provide a biologically safe level of 
artificial gravity for its human crew members. All of the above MTV design options have 
implications concerning the complexity, size, and hazards of assembling, integrating, and 
checking out MTVs. It may take over a year to assemble and prepare such a craft for launch 
towards Mars. If robots are not used, it will take enormous manpower to perform these 
tasks. Large-scale application of robots is essential for MTV assembly, integration, and 
testing in space. 

Robots will be an integral part of the MTV crew, ready at a moment’s notice to carry 
out emergency EVA in interplanetary space or in orbit about Mars, especially if nuclear 
propulsion systems are employed. 

Operations on the Lunar and Martian Surfaces 

The few who may venture to the Moon or Mars in the next quarter century will not be 
able to carry out all the construction tasks on the lunar or Martian surfaces required if man 
is to establish a permanent presence on the Moon, travel to Mars, and return safely and in 
good health to Earth. Even before man ventures to those planets, robot probes will be needed 
to explore and map the lunar and Martian surfaces. Robots will be especially needed to 
construct a permanent Moon base that includes habitats, radiation protection systems, 
surface power sources, and cargo and space vehicle processing facilities. Lunar Transfer 
Vehicles (LTVs) will have to be serviced in areas without radiation protection, in preparation 
for return to Earth. Finally, if a true local industrial economy is to be developed on the 
Moon, robotic resource extraction and processing equipment will be needed to generate 
oxygen, water, rocket fuels, and perhaps other products. 

On the surface of Mars, space vehicle processing, testing, and perhaps repair 
operations will likely be at least partially carried out by robots. Robot rovers and other types 
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of mobile robots will be used to extend the exploration activities of humans on the Martian 
surface. 

CLASSIFICATION OF ROBOTS 

Robots can be classified in the following general categories: 

• structured task robots 

• teleoperated robots 

• semi-autonomous and autonomous robots 

Below we discuss these categories in more detail. 

Structured Task Robots 

The simplest and most prevalent robots are Structured Task Robots (STRs). STRs are 
used widely in the automobile, electronics, and semiconductor industries. The largest 
concentration of STRs is in Japan, where most of the world’s STRs are now made [Refs. 2, 3], 
A typical STR is immobile and consists of a six-degree-of-freedom robot arm and a two-to six- 
degree-of-freedom end-effector. STR actions are completely programmed into an associated 
computer which scripts the robot’s motion and manipulation activities. STRs will typically 
execute the same motion and manipulation script over and over again in a precisely 
controlled manner. 

For an STR to perform useful work, it must be carefully integrated into the 
manufacturing or assembly process. Components that it manipulates must typically be 
oriented in a single direction and located in one correct position. If a component is left in the 
wrong position, the STR may damage it during manipulation or welding. The entire 
production line can easily be disrupted by one STR not doing its job correctly. 

One of the most important applications of STRs is in semiconductor industry “clean 
rooms” where Very Large Scale Integration (VLSI) and Very High Scale Integration Circuit 
(VHSIC) chips are fabricated. High-density integrated circuits contain circuit patterns so 
small that almost all dust particles must be removed from clean-room air. Otherwise, 
circuits will be “smudged out” during fabrication. For example, in the NMB Semiconductor’s 
state-of-the-art class 1 clean room, 2 where 4 Mb Dynamic Random Access Memory (DRAM) 
chips are fabricated, extensive use must be made of robotics because humans would 

2 Meaning on average there is only one 0.5 micron size dust particle per cubic foot of air. This is 
one million times cleaner than typical air [Ref. 20]. 
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contaminate chips in fabrication [Ref. 19]. This difficulty is made even more apparent when 
it is realized that the average human being exhales an average of 500 1-pm particles with 
every breath. Future generations of high density integrated circuits at crucial junctures in 
the fabrication process will be made completely by STRs. 

Teleoperated Robots 

The second class of robots is Teleoperated Robots (TORs). TORs are remotely 
controlled by a human controller, who receives feedback information from the robot and 
transmits command information to it via radio link or wire. Because TORs are controlled by 
humans, they can work much more flexibly and in more complex, unpredictable work 
environments. TOR motions and manipulation capabilities are directed by the human using 
a control interface like a joy stick or keyboard. An ideal TOR — from the controller's point of 
view — would have arms and hands that were a perfect substitute for a human's. In order to 
establish this type of control, high fidelity TOR control interfaces are required. Recent 
research and new technology developments which bear on this question will be discussed in 
some detail later in this Note. Finally, it should be mentioned that one of the primary 
advantages of employing TORs is that the robot can work in an environment hazardous to 
humans. TORs can extend human presence into regions or environments deadly to man. For 
example, TORs are used extensively in the commercial nuclear power industry in high- 
radiation environments. TORs are also used in deep under-water exploration and 
construction work [Ref. 21]. 

Semi-Autonomous and Autonomous Robots 

A semi-autonomous robot is one which can perform many activities in its work 
environment without human intervention or guidance. Semi-autonomous robots are 
sometimes called telesupervised robots if they are controlled intermittently by a remote 
human controller. A good example of a semi-autonomous robot is the NASA/JPL Magellan 
space probe. It can perform many work activities autonomously, a capability that is needed 
for probes which may be located so far away from Earth that a significant time delay is 
incurred in communication. However, semi-autonomous robots are vulnerable to 
unanticipated or unpredictable changes in their operational environment. That is, they are 
autonomous only within their prescribed work environment or mission profile. This 
vulnerability was recently illustrated when NASA temporarily lost contact with the Magellan 
space probe. Some unanticipated event occurred in its operating environment, causing the 
probe to shut down and go into a safe mode. 
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An STR could be considered a semi-autonomous robot according to the simple 
definition given above. However, most STRs have no sensing capabilities and no world model 
(a model of its environment stored in its memory). Instead an STR has internalized only a 
prescribed set of actions. Here, autonomy or semi-autonomy shall refer to the capability to 
compare an internal world model with external stimuli collected by the robot’s sensors, as 
well as the ability to work without human intervention. 

What is an autonomous robot? This is a surprisingly difficult question to answer 
because it invariably refers back to capabilities ascribed to humans. Most humans are not 
autonomous in all their activities. They require help or guidance to do their jobs. An ant in 
an ant hill or following an ant trail searching for food appears autonomous. A chemist in a 
laboratory using equipment from a distant vendor is not. His tasks could not be completed 
without the equipment and “brain power” encapsulated within the chips of that equipment. 
Autonomy, viewed formally, depends on the context. 

The ultimate autonomous robot would be able to take the place of a human and 
perform any physical or cognitive activity humans are capable of. In other words, for a fully 
autonomous robot, any possible human task should be equivalent to a set of robotic tasks. 
Therefore, to define robotic tasks for an autonomous robot one can use human activities as a 
model. 

Although the concept of the autonomous robot was introduced sometime ago, such 
robots are still far from being realized and may never truly be created by human beings. 
There are a host of unsolved problems in robotics which make it appear that machine 
autonomy will not be achieved in the foreseeable future. For example, robots cannot “see” 
and understands images like humans do. Nor can they perform complex mechanical tasks in 
an unstructured, unpredictable work environment. Perhaps the real crux of the matter is 
that robots, even if equipped with artificial intelligence software programs or expert systems, 
are not capable of reproducing all the rich and varied cognitive decisionmaking processes the 
typical human can. Machines may win at chess, but they cannot think — yet. 

STRUCTURED AND UNSTRUCTURED ENVIRONMENTS 

As was discussed above, it is important to define the context or work environment of a 
robot or a human so that either entity could be considered to be autonomous. However, 
human work environments need not to be precisely defined for a human worker to function 
autonomously. Thus, typical human work environments are said to be unstructured. 

It is relatively easy to describe an unstructured work environment. Imagine a typical 
auto repair shop cluttered with a wide assortment of tools and spare parts. Tools, good parts, 
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and trash may be put in no particular order. A novice mechanic would have to search 
carefully for every tool or part he needed. Because each type of auto is built to different 
specifications, the mechanic has to carefully devise the set of operations needed to replace a 
brake shoe, for example. 

A mechanic, and certainly the designer of a robot work environment, can make it 
easier on the worker if the work environment is structured in a simple, orderly way. A 
veteran mechanic may, to ease the burden on his memoiy, develop a system to store his tools 
and spare parts. Wrenches go in one drawer, old hoses in a particular box, etc. The veteran 
mechanic remembers approximately where most things are (he has constructed a global or 
world model of his work environment). 

Every now and then the mechanic may forget where something is and have to search 
for that one essential missing wrench. Because his work environment is not completely 
structured, he has to devise a search strategy and carefully look behind, under, and inside 
other items in the shop. In short, he behaves as if he is in an unstructured work 
environment and resorts to his full global sensing, navigation, and movement capabilities. It 
is these latter capabilities that are so difficult to emulate in a robot. 

Consider a robot designed to repair cars in the partially structured work environment 
such as that of the veteran auto mechanic. Inside this robot’s memory would be a model of 
its work environment which included the approximate position, number, and type of 
wrenches in each drawer of the tool cabinet, the position of spare parts, etc. This robot would 
move about the repair shop by following pre-programmed paths to the objects it needed. 

Such a robot would also have local sensing, navigation, and movement skills to pick up 
and use tools. Its robot hand would have to orient itself correctly to grasp a particular 
wrench. It might also have to tuck its arm in close to its body when it moved about the shop 
to avoid hitting obstacles. Finally, it would have to align whatever tool it had in its grasp 
and guide it into position to assemble or disassemble the car it was working on. The robot 
would have to perceive the position of its own hands, the tool in use, and the object to be 
manipulated. Still, an event such as the robot finding one particular wrench out of place may 
stymie it. It would suddenly find itself in an unstructured work environment and would 
have to search the entire shop for a wrench without the benefit of its tool location database. 

A robot that can find a lost wrench is more appropriately called a semi-autonomous 
robot. Such robots cannot yet be built with current computer vision and expert systems. On 
the other hand, a robot designed to operate in a highly structured environment could be of 
considerably simpler design than an autonomous robot and could be built today. It would not 
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have to be capable of independently constructing search strategies for lost objects or 
constructing arbitrary navigational paths through the clutter of a messy repair shop. 

A robot’s work environment can be refined still further by designing each task to be 
executed by the robot. If each elementary robot task is defined by precisely positioning all 
tools and parts to be manipulated and the object to be repaired or assembled, the work 
environment is said to be structured. If the essential geometrical characteristics of this work 
environment are inserted into the robot’s memory and each robot movement is precisely 
programmed, this robot is essentially an STR. An STR does not require any of the global or 
local sensing characteristics described in the previous section. All the actions of an STR are 
prescribed in advance by its program. 

How could one turn an auto repair shop into a completely structured environment? 
Every mechanic’s tool would have to be precisely aligned to the model of work environment 
programmed into a STR’s memory. Every spare part would also have to placed in a position 
coinciding with the STR’s model of the environment. Nothing could be out of place with 
respect to this model. Using the position coordinates of all these objects as inputs, each 
specific robot action or work task could then be programmed into the robot’s memory as well. 

If one considers that several hundred tools may be needed to repair a single type of 
car, that a car is composed of many hundred parts, and that each robotic repair task may be 
composed of many discrete subtasks in which several tools and parts must be manipulated, it 
becomes apparent that an auto repair STR would require an enormously large memory. Its 
library of software codes would have to be quite large as well. The more complex the tasks 
an STR is programmed to perform, and the more complex its work environment, the larger 
and more sophisticated its computer hardware and software must be. 

Nevertheless, STRs and structured work environments can provide an enormous 
benefit. If a robot’s work environment and each robot task can be precisely defined, then, in 
principle at least, a well-defined software code can be written to govern an STR’s actions. On 
the other hand, the software codes required to enable a robot to sense, navigate, and move in 
a work environment that cannot be modeled in a static fashion are much more conceptually 
complex. Such software codes must be based on general navigational algorithms that accept 
real-time inputs from the robot’s sensors, and which instruct the robot’s limbs or wheels how 
to move in real time. While an STR’s software and hardware can easily be sized to operate in 
real time (because it operates using a static database), a robot operating in an unstructured 
environment must possess an operating speed margin to deal with the variable size and 
complexity of a dynamic database continuously being updated by the robot’s sensors. 
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It should be noted that robotic software for operation in an unstructured environment 
is also more complex in other ways. Humans navigate and move in complex environments 
based on higher-order commands, such as “move over to the car’s left rear wheel and take off 
the hub cap.” Creating software capable of accepting such higher-order commands (or input 
instructions) is an extremely challenging task, and one which is as yet unsolved for 
navigation or locomotion systems. 

How can current robot technology be used to support SEI operations in space and on 
the lunar and Martian surfaces? The above discussion of an auto repair shop work 
environment indicates how difficult it may be to develop autonomous robots to assemble or 
repair spacecraft in orbit. SEI assembly and repair work environments must be structured 
in as simple and as orderly a fashion as possible. Construction and assembly processes must 
be precisely defined — down to the last nut and bolt — and designed from the start to be 
performed by robots. 
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III. DISCUSSION OF SUBMISSIONS 


Project Outreach submissions that scored highly were grouped according to several 
categories that represent the key A&R issues raised during review and analysis. The A&R 
issue categories are: system integration of robots and SEI systems, STRs and structured 
work environments, TORs and unstructured work environments, TOR control interfaces, SEI 
robot mission planning, and the transition to autonomous robots. 

SYSTEM INTEGRATION OF ROBOTS AND SEI SYSTEMS 

If robots are to assemble, inspect, and repair SEI systems in space, on the Moon, or on 
Mars, they will have to be compatible with SEI systems. To ensure compatibility, SEI robots 
and systems must be designed systemically. This design activity will be exceptionally 
challenging because comprehensive design rules must be developed for a large number of SEI 
systems, subsystems, and robots. Major SEI systems will be built by an array of U.S. 
contractors and potential international partners. System design efforts at all SEI contractors 
must track each other, and, through a consensus process overseen by NASA, systemic design 
standards must be developed. SEI robotic system design is further complicated because the 
field of robotics is still in its infancy. Nevertheless, the potential payoff of integrating robotic 
technologies with SEI systems is great. Truss assemblies, solar panels, habitats, aerobrakes, 
propellant tanks, nuclear power reactors, and many other SEI systems could be efficiently 
assembled, inspected, and repaired by robots if such standards were adopted. 

Systemic design standards for SEI robots, work environments, and systems can only 
be achieved by capturing and maintaining configuration control over all SEI system design 
data. Large volumes of detailed engineering design data must be captured in a common 
digital format and made portable so that it can be used by different system contractors 
during design and manufacturing, and by robots in space during assembly and repair 
operations. Automated system design capture has been made possible with the advent of 
integrated Computer Aided Design/Computer Aided Manufacturing (CAD/CAM) tools. 
CAD/CAM tools are now used extensively in the semiconductor industry to design, simulate, 
and manufacture advanced integrated circuits such as Application Specific Integrated 
Circuits (ASICs). 

Design of a particular ASIC is accomplished by combining the designs of several 
smaller elementary circuits (standard cells) onto a single chip. The standard cell designs are 
integrated together and adjusted so that the overall chip functions as intended by means of 
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computer simulation. This system integration process has proven very cost-effective because 
a particular contractor can develop a limited standard cell library which can be used to 
design and manufacture a wide array of ASICs. Development time and costs for many newly 
ASIC designs can be greatly reduced. To speed this process even further, newly distributed 
CAD/CAM products and services are now being offered that tie together the ASIC customer 
and manufacturer into a single network, thereby reducing the number of design iterations 
even further [Ref. 22]. 

Transportability of CAD/CAM files is also being improved with the introduction of 
commercial standards such as the emerging Electronic Data Interchange Format (EDIF) 
CAD/CAM standard [Ref. 1], However, because of the many different CAD/CAM system 
vendors and platforms in use in industry, a great deal of non-interoperability between 
CAD/CAM systems remains. NASA should monitor the development and use of CAD/CAM 
tools and standards in the semiconductor and other industries and adapt these increasingly 
powerful design tools to SEI systems and robots. NASA should consider adopting a single 
CAD/CAM standard of its own to be used in the design, simulation, and manufacturing of all 
SEI systems. 

The digital capture of SEI system design knowledge will offer a tremendous benefit to 
robot assembly and repair operations. Imagine a CAD/CAM file of an SEI system such as a 
gas turbine generator in a dynamic power system. This CAD/CAM file and associated STR 
repair commands (similar to those used to manufacture the system on an STR assembly line 
on Earth) could be loaded into an STR tasked to repair the turbine generator in space. Once 
the STR was loaded with this data, it would “know how” to take apart the generator, and 
given appropriate intervening instructions, “how to” replace the faulty part with a new one. 
Similarly, if a comprehensive CAD/CAM database were available for SEI systems, a TOR 
operator could call up the original design drawings for the particular system he or she was 
assembling. In a more advanced autonomous robot, CAD/CAM files could be loaded via a 
local area network connection directly into the robot’s memory. The autonomous robot’s 
expert systems could then draw upon CAD/CAM data during assembly or repair operations. 

However, as alluded to earlier, many difficult technical problems are as yet unsolved, 
and extensive research will be needed to create highly autonomous robots. Consequently, 
many SEI construction, inspection, and repair tasks must be designed to accommodate the 
limitations of current robots. Even twenty-five years from now, many SEI construction tasks 
or work environments will still be too complicated or demanding for robots, and humans will 
still have to perform them. One of the long-term objectives of an SEI R&D program should 
be to minimize the number of man-hours needed to perform dangerous, repetitive, or 



- 18- 


physically demanding tasks. Robots can free astronauts from the drudgery of space 
exploration and enable them to devote their valuable time to scientific research, and the 
search for new resources, and to report what they see and discover on other worlds of our 
solar system. 

Task Allocation Among Humans and Robots 

Given the capabilities of robots developed to support SEI, individual assembly, 
inspection, and repair tasks must be evaluated to determine those that should be performed 
by robots and those that can only be accomplished by humans. 

• Task Allocation Among Humans, Teleoperated Devices, and Robots 
(#101440) 

Submission #101440 is a proposal to develop a methodology for systematically 
allocating tasks between robots and humans. Reference is made to robot and human task 
allocation studies performed in the nuclear power industry (an industry where the most 
sophisticated commercial robots are now used). A specific four-level approach is outlined in 
the submission. Because a backup document was not included, it is impossible to analyze in 
detail the specific approach advocated. Nevertheless, this submission was given a relatively 
high score because the problem it addresses is so important for systemic integration of SEI 
robots and systems. It is also a top-level system design task that must be addressed at the 
outset and executed in parallel with other SEI development activities. 

The submission authors point out that certain types of robots, such as TORs, must be 
closely controlled by humans. TOR Command and Control (C2) can be extremely demanding 
and fatiguing for human operators. Thus, human capabilities must be considered carefully 
when allocating tasks to TORs and TOR controllers. TOR task allocation depends on the 
task to be performed by the robot, controller skill level, and the nature of the robot control 
interface. A well-designed methodology for allocating tasks among robots and human 
workers will significantly enhance the productivity of SEI robots and astronauts. 

STRs AND STRUCTURED ENVIRONMENTS 

If SEI systems and robotic construction processes are systematically developed, the 
capabilities required of corresponding construction robots can be simplified considerably. On 
the other hand, if SEI systems and construction processes are not so designed, assembly 
robots would have to possess sensing, manipulation, and navigation skills similar to those of 
a human being. Because such autonomous robots are far beyond the current state of the art, 
this type of robotic assembly would not be possible in the near term. Furthermore, 
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development of highly capable autonomous robots for SEI will be very costly. It is therefore 
important to constrain where possible the design of SEI structures and construction 
processes so they can be executed by robots within the current state of the art. One 
important way this can be accomplished is by developing highly structured environments for 
the construction process. 

• Robotic Assembly of Large Lunar Structures (#100378) 

This submission describes a robotic truss assembly system designed to operate in a 
highly structured environment. It is designed to assemble trusses in Earth orbit using a 
relatively simple commercial robot arm. The arm is programmed to pick up truss members 
from a bin and snap them into place in the truss. It is mounted on a planar X-Y motion base 
and a separate turntable platform. The arm can be arbitrarily oriented within a six by six 
meter area. 

This robot system could be extended to perform a number of repetitive assembly tasks. 
If the system were moved as the construction process proceeded, large trusses or other 
periodic structures could be assembled. If SEI structures and construction processes are 
carefully defined, current industrial robotic technology can be used in the assembly process. 
Furthermore, because assembly occurs in a highly structured environment, the assembly 
process could be completely automated as system software is further developed and refined 
(the system could be made completely autonomous within its structured work environment). 
This type of robotic assembly system could therefore significantly reduce manpower 
requirements for SEI construction tasks. 

The development of precisely these types of STRs and STR structured work 
environments has enabled the Japanese to continue to increase the productivity of their 
automobile and semiconductor industries and reduce associated manpower requirements to 
levels significantly below those of their international competitors. Similarly, development of 
STRs and STR structured work environments for SEI, as suggested in submission #100378, 
can significantly reduce associated SEI manpower requirements in space and on the surface 
of the Moon or Mars. 

The system proposed in submission #100378 is being developed for low Earth orbit 
applications, such as space station construction. The submission authors propose that this 
system be adapted to construct large structures on the Moon. The suggestion is appealing in 
general terms; however, this particular system may have to be adapted to work effectively in 
the gravitational field of the Moon. Because trusses bend under the influence of gravity, the 
construction process and work environment may have to be modified. In particular, the 
positions through which the robot arm cycles during construction may have to be ac^usted 
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sequentially to take into account truss deflections. While it is certainly possible to prepare 
lunar construction sites so the robotic assembly system can function in a highly structured 
environment, it should be pointed out that there are lunar construction tasks which cannot 
be performed in a structured environment, such as excavation of lunar soil or transportation 
of construction materials over lunar terrain. The latter activities will require robots with 
more sophisticated capabilities. 

TORS AND UNSTRUCTURED ENVIRONMENTS 

Not all SEI robot tasks can be designed so they can be executed in a structured 
environment. For example, during the initial stages of lunar base construction the lunar 
surface will be unprepared and may present unforeseen obstacles and problems for human 
and robot construction crews. The position, orientation, and shape of construction materials 
may vary so far from the assumed norm that STRs cannot be programmed to reliably 
manipulate them. Robots with more sophisticated sensing, navigation, movement, and 
manipulation capabilities will be needed to operate in such unstructured environments. 

TORs, because they can be controlled by humans and make use of human sensing, 
navigation, and movement capabilities, can potentially operate in such unstructured 
environments. The human TOR controller is presented by means of communications and 
display devices with visual and kinesthetic sensor information from the robot, which can 
reveal unanticipated objects in the robot’s environment. The human controller can therefore 
alter the robot’s actions in real time (or near real time, depending upon the communications 
time delay) to compensate for the lack of structure in the work environment. 

Submissions #100695 and #100338 propose to develop TORs which would be well 
suited for such tasks. Both submissions were given high scores because the proposed robotic 
systems are potentially capable of replacing humans in EVA construction or repair tasks. 
Astronaut EVA is potentially hazardous and requires long decompression times. Any robot 
which can reduce the necessity for human EVA will have high utility for SEI. The extent to 
which robots can replace humans in unstructured work environments depends upon how 
closely they can reproduce the functions of the human hand, how well they can be controlled, 
and how versatile they are. In regard to the first point, both robots can use anthropomorphic 
end-effectors compatible to a large degree with the human hand. Because both robot arm 
systems are controlled with mimetic exoskeletons, the robot arms can, in principle, easily be 
controlled to avoid obstacles, pick up tools, and apply carefully measured torques to bolts, 
nuts, or screws. 
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Both dual-armed TORs resemble the FTS and would be capable of operating in a 
dynamic and complex work environment. Also, like the FTS, both robots would be immobile 
and would have to be hauled to a surface work site by crane, or to a work space station site 
by the space shuttled Remote Manipulator System (RMS) or a similar system. 

• Space Robotics: A Highly Dexterous Robot with Adaptable Control 
Strategies (#100695) 

This submission proposes developing a dual-armed anthropomorphic TOR called the 
Astronautics Dexterous Anthropomorphic Manipulator (ADAM2). ADAM2 will have 
interchangeable hand-wrist packages to increase system adaptability. The primary ADAM2 
end-effector has been designed to an anthropomorphic end-effector design rule. It has three 
fingers and has been designed to nearly reproduce the force and grip of the human hand. 

Such a design rule is highly desirable for SEI operations because a robot equipped 
with anthropomorphic end-effectors could potentially perform unanticipated tasks, such as 
emergency EVA, and take an astronaut's place in dangerous situations. As stated in the 
submission, an anthropomorphic design “minimizes the need to restructure tasks and work 
sites, and it allows human operators to control and train the system quickly.” ADAM2 
control is effected by a mimetic exoskeleton and zero-motion master fitted over the 
controller's arms and hands. This submission will be analyzed in greater detail below when 
TORs are discussed. 

• EVA Equivalent Space Telemanipulation System (#100338) 

This submission describes a similar dual-arm anthropomorphic TOR controlled by an 
exoskeleton dual arm-hand master controller. Its primary end-effectors are also designed to 
be compatible with the human hand. In addition, this TOR can be equipped with “a host of 
EVA tools, power tools, auxiliary and special purpose devices to perform many tasks.” The 
submission does not describe these tools in detail, but presumably most of them would also 
be usable by a human EVA astronaut. 

Robot End-Effector Design Rules 

Submission #100378 illustrates an important potential design rule for SEI STR robots 
and SEI structures. The STR end-effector and truss members have been specifically 
designed so the arm can easily manipulate these truss members and lock nuts (the latter 
fasten truss members together). Other structures could be made compatible with this end- 
effector and could be manipulated by the same system. Comprehensive robot manipulator 
standards or design rules should be developed that include end-effectors, SEI structural 
components such as trusses, lock nuts, and the terminal wrist connectors of robot arms. 
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Then an end-effector could be taken from one type of SEI robot arm and used on another 
system. In the same way, one end-effector could be used to assemble several types of 
structures. Such a design rule would minimize the number of different end-effectors 
required for assembly and repair tasks. A single end-effector design could probably not 
perform all assembly tasks, and a set of differing sizes and capabilities will most likely be 
required. 

The submissions above (#100378, #100344, #100338, and #100695) indicate the need 
for one anthropomorphic and one or more non-anthropomorphic end-effector design rules. A 
research program that cuts across all NASA and university programs in space robotics, space 
structure construction, and repair research is probably needed to establish a comprehensive 
set of standards and to eliminate needless duplication in robot end-effector development. 

• The Moon-Mars Autonomous Resource Management System (ARMS) 
(#101469) 

This submission presents a broad conceptual design for a family of SEI robots capable 
of performing the following tasks working in an unstructured lunar or Martian environment: 
remote surveying, facility construction and repair, transportation or installation of 
equipment, mining, and the handling of hazardous materials. In addition, ARMS robot 
rovers could be configured to provide SEI power or communications mission support. ARMS 
systems would be of modular design and could be reconfigured in the field to perform the 
different tasks mentioned above. 

The ARMS proposal is notable for its emphasis on development of an integrated 
overall SEI A&R infrastructure and for its attempt to describe how this A&R infrastructure 
can be integrated with other SEI infrastructure elements (in particular, communications and 
surface power). There are three key elements to the A&R infrastructure proposed: Earth 
facilities, a LEO depot, and Moon-Mars facilities. These elements would be designed in an 
integrated fashion to facilitate transfer of digital data, spare parts, fuel, and materials 
between facilities. 

Robot mission planning, simulation, testing, and teleoperational or telesupervisory 
control would be performed on Earth at the ARMS ground facility. Later this ground facility 
could be used by private or corporate users to pursue commercial ventures or scientific 
research at SSF, on the Moon, or on Mars. 

The LEO depot, which would be located at SSF, would serve primarily as a logistics 
base and TOR communications node in support of lunar or Martian ARMS systems. ARMS 
Moon-Mars facilities would be comprised of two systems: a Transportable Service Rover 
(TSR) and a Fixed Depot Station (FDS). 
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The ARMS TSR would function independently of the FDS, but it is envisioned the two 
systems would be used together to extend the range and types of ARMS missions. The 
ARMS TSR would be equipped with standard power and controller interfaces so it could be 
easily reconfigured with power sources, data management modules, sensors, expert system 
computer modules, manipulation arms or shovels, etc. All these component subsystems 
would be standardized. These suites of equipment would all be designed so TSRs could 
cooperatively reconfigure themselves without direct human intervention. 

The ARMS FDS would serve primarily as a protective berthing storehouse for TSRs 
and as a forward base for data management and man-tended operations. The FDS could also 
remotely control TSRs. The FDS would be comprised of storage facilities, power supply and 
distribution, computer, communication, and display and control equipment. 

The ARMS concept lacks sufficient engineering detail in its present form, but reference 
is made towards evolving from TORs to autonomous rovers and depot stations. Hardware or 
software complexity of ARMS systems is not described; however, reference is made to 
previous NASA and Canadian robotic programs and tentative cost estimates are presented. 
The ARMS conceptual design is not new, but the authors have discussed standardized and 
modularized A&R concepts well and have shown how such systems may be integrated into 
SEI architectures. The technical feasibility of achieving the autonomous or semi-autonomous 
robots envisioned is not addressed. 

Robot Locomotion, Stabilization, and SEI Systems 

Space construction robots will have to move about and fix themselves to SEI space 
facilities or support structures. These robots could be free-flying, equipped with their own 
propulsion systems, “walkers* which move mechanically by using legs with specially 
designed cleats, or immobile like the FTS. 

All these robots, whether mobile or immobile, must be equipped with grips or cleats to 
stabilize themselves on space structures. System engineering design rules are needed which 
establish “scars” [Refs. 1, 16, 17] for SEI space facilities to ensure compatibility with robot 
stabilizers. No submissions were received in this area. 

While several submissions propose various locomotion systems for exploration robots, 
no highly ranked submissions were received that describe locomotion systems for space 
construction robots. In the literature, locomotion systems which exploit properties of 
structured environments, such as rails, have been proposed. Such “structured locomotion 
interfaces” could be developed and standardized to enable STRs to replace humans in simple 
EVA activities. 
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Robot Vision and SEI Systems 

Space and surface structures to be assembled by the autonomous robots could be 
designed to permit robot vision systems to quickly recognize structure type and orientation. 
Bar codes similar to those used in the retail industry have been suggested by many in the 
literature [Ref. 1], The performance of bar-code-reading laser scanners in a LEO or lunar 
surface environment should be evaluated. Bar code symbol size, laser wavelength, and solar 
and lunar noise levels are among the factors to be considered. Bar codes could also be useful 
for inventory control of space systems (e.g., a lunar base) even if autonomous robots are not 
used. Unfortunately, robot vision systems may still be too primitive to permit systematic 
study of alternative robot bar code and object recognition techniques. Indeed, the theory of 
mammalian visual processing is still in its infancy [Ref. 23]. No submissions were received 
in this area. 

AUTOMATED SYSTEM MONITORING 

Automated system monitoring can enhance mission safety and system reliability. 
Automated system monitoring is practiced to a great extent today in certain aerospace fields, 
such as rocketry. Continued advances in digital microelectronics and recent developments of 
analog microsensors may make many new types of smart components (systems with 
embedded automated system monitoring equipment) feasible and cost effective. 

• Smart Components (#101324) 

This submission proposes the widespread use of smart components in SEI systems. 
The submission is very general and brief. However, its potential advantages were so 
significant that the reviewers were compelled to score it highly. Smart components offer 
several potential advantages, especially if they can be made small and light enough so as not 
to significantly affect overall system design requirements. 

Smart components can reduce the probability of catastrophic or initial component 
failure. They can collect useful data on component performance parameters and increase 
component life (by signaling when component failure is imminent and enabling changes to be 
made in system performance to preserve component life). 

However, smart components present several challenges to SEI system designers by 
placing additional requirements on ancillary communications and Automated Data 
Processing (ADP) systems. Smart components must be linked via local area networks or 
communications buses to display systems, databases, or monitoring expert systems. 
Widespread deployment of smart components could greatly increase the communications 
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burden on local Data Management Systems (DMSs) and even perhaps on interplanetary SEI 
communications networks. Expert systems must be developed to interpret, manage, and 
filter smart component readouts to reduce the burden on mission crews or remote ground 
controllers. These downstream requirements imply that deployment of smart components 
should initially be limited to critical subsystems, such as life support, and their introduction 
should be coordinated with associated expert systems and communications networks or 
buses. 

ROBOT CONTROL INTERFACES 

Robot control interfaces can take many forms. Digital interfaces such as keyboard 
commands are used frequently in academic research. Joysticks can be used to control the 
orientation of a robot arm. Mimetic exoskeleton gloves can provide tactile and force 
reflection feedback. Autonomous robot control could be effected by speech commands and the 
presentation of visual graphical information to the robot, much like command and control of 
human workers would be accomplished. 

Autonomous Robots would be the most desirable type of SEI robot because carefully 
designed work environments and many SEI system hooks and scars would not have to be 
developed (as required for STRs). Moreover, the additional manpower and communications 
requirements for TORs would not be needed. But versatile Autonomous Robots (AR), capable 
of operating in unstructured SEI environments such as a planetary surface or in free-flying 
LEO will require many sophisticated and as yet unrealized capabilities. As a consequence, 
IOC dates for Autonomous Robots cannot be predicted and may not be achievable within the 
timeframe of SEI. 

Feedback mechanisms, by which an operator ascertains the orientation of the robot 
and its work environment, are key to the interface used to control the robot. A number of 
Project Outreach submissions were received that describe specific robot control interfaces or 
propose development of new highly capable ones. Most of these submissions consider 
development of more capable TOR control systems. 

Teleoperated Robots (TORs) 

• Space Robotics: A Highly Dexterous Robot with Adaptable Control 
Strategies (*100695) 

A team of researchers proposes to extend an existing robot program for a single-arm 
robot, the Astronautics Dexterous Anthropomorphic Manipulator (ADAM), into a robotic 
system with dual, cooperative arms (ADAM2). ADAM2 would be relatively low mass (one 
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arm unit < 40 kg), modular, accept many types of end-effectors, provide force feedback to the 
user, and accommodate increasing supervisory-mode operation in defined tasks. Mobility, 
power, and control means are not described. Its size, tip speed (> 50 cm/sec), and seven 
degrees of freedom would allow ADAM2 to provide anthropomorphic motions and allow new 
users to become skillful in a short time. A standard approach is described for the 
development of ADAM2. 

ADAM2 types of telerobots should be vigorously developed because they can permit 
humans to work in space from inside a space vehicle. In addition, telerobots will permit 
many tasks within cislunar space to be done by workers on Earth via telemetry. 
Anthropomorphic robots should be viewed as one of a spectrum of robot types ranging from 
relatively simple units (e.g., a roving TV eye) to those that are large, powerful, and 
specialized (e.g., heavy duty excavators and construction robots for lunar operations). 

• EVA Equivalent Space Telemanipulation System (#100338) 

FTS, now under way, will be a large version of the dual-arm space Telemanipulation 
System (TMS) described in this submission (< 100 kg, < 150 W). The TMS, brought to the 
work place by a boom or rover vehicle, is operated by a human in a pressurized environment 
via an exoskeleton dual arm-hand master controller. The operator also positions and 
controls the TMS support mount or vehicle. The end-effectors can be “hands” or special 
power tools. A test version of the device will fly in the early 1990s in the shuttle bay. It 
could be adapted for use in space or in protected environments on the Moon. There is no 
detailed discussion of problems of TMS utilization on the Moon as would be encountered with 
lunar dust fouling up bearings and contaminating manipulators or tools. 

TMS-like devices would be extremely useful both inside and outside the space facility. 
Many small contracts should be let to provide a suite of robots of the size of TMS. Those 
robots should compete in many demonstration tasks, both to select the viable approaches and 
to stimulate thinking about how to use robots in space. Much work remains to be done 
beyond the design extension proposed in this submission. 

• Telerobotics in SEI Surface Operations (#100341) 

The summary and backup paper for this submission provide a good qualitative account 
of some of the uses and advantages of teleoperated robots in the exploration of a region on 
the Moon or Mars and for the emplacement of initial habitats. Early emplacement tasks 
would include site preparation, placing habitats in revetments, covering the habitats with 
soil for protection against galactic and solar cosmic rays, and mining for water-ice. There is 
virtually no engineering or technological data given. The broader implications of telerobotics 
for system design and mission operations are not discussed. 
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The paper draws extensive analogies between Moon and Mars SEI efforts and the 
1700s and 1800s exploration and settling of the American frontier. Considerable emphasis is 
given to locating and extracting water on Mars. The analogies are likely to divert SEI R&D 
personnel from the primary challenges of human exploration and settlement of the Moon and 
Mars. In fact, NASA must aggressively move to achieve effectively complete recycling of 
water, carbon, nitrogen, and other life chemicals. Efficiently acquiring energy and rejecting 
waste heat to operate the recycling means must be the major engineering achievement if 
humans are to be supported beyond Earth. Water would be a “once mined” quantity and 
replacement kept to a very low level. Similarly, there is a picture of humans roaming about 
the Moon and Mars in the style of an early American prospector. This simply will not be the 
case. Exposure to galactic and solar cosmic rays and their induced products will severely 
limit the integrated, long-term presence of humans on the surface of either the Moon or Mars 
or outside a space facility. The importance of telerobotics in support of permanent human 
presence in space cannot be overemphasized. 

One-of-a-kind robots, either autonomous or teleoperated, will be very expensive. 

Viking was not a cheap mission, had only limited telerobotic capability (1 meter reach, 
grasping, and placing), and provided only limited scientific return, especially with respect to 
determining the presence of life on Mars. Robots become very attractive in the absence of 
other choices or when they offer economies of scale. The latter alternative is not covered in 
this submission but has been addressed in the ARMS submission discussed earlier 
(# 101469 ). 

If telerobots can be equipped with interchangeable suites of sensors, lunar and 
Martian environments could be examined remotely and scientific or engineering research 
carried out. Such detailed remote experience of the environment can allow the operator to 
program the machine to perform operations repetitively by taking the machine through the 
sequence of moves. Teleoperation can be effective from Earth to orbit and in some cases, via 
shared control, out to the Moon. Scientists on Earth could be in close contact with 
experiments in LEO space facilities or with a processing plant on the Moon. Teleoperated 
robots could also perform field geology on the Moon or Mars. 

Telerobots can be useful even when there is a long time delay for communication 
between the machine and operator via shared control. The teleoperated robot can conduct 
automatic routine operations and the procedures can be revised post real-time by the distant 
operator. Teleoperation from Earth to Mars would entail very long time delays, exceeding 10 
minutes in the best case. In the Earth-Mars case, teleoperation would best be used to 
reprogram from Earth the complex local activities of a stationary robot on Mars. 
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There is always a need for placing many trained observers in a remote new area. 
Teleoperation offers a means to do so without risking people or entailing the great expenses 
associated with people. In addition, several teleoperated vehicles could be placed in a new 
region. Such vehicles could be designed so they could be taken apart, to some level of 
assembly, by one another. Thus, they could form their own spare parts pool (as described in 
#101469). 

Telepresence 

Present-day TOR control can be a fatiguing and difficult task for the operator. 

Because the operator must frequently interpret low-resolution feedback data from the robot’s 
sensors, he may be forced to proceed at a very slow pace when directing the robot’s limbs or 
end-effectors. If he swings the robot’s arm too quickly and misinterprets or doesn’t receive 
necessary feedback data, the arm may knock over or damage nearby objects. Lack of high 
fidelity TOR feedback can reduce operator productivity to the point where a human could 
perform the same tasks many times faster than the TOR This limitation has prompted 
researchers to develop and investigate operator interfaces capable of presenting a more 
realistic and detailed representation of the robot’s environment. Such high-resolution 
interfaces are said to provide telepresence to the TOR operator. 

Present TORs, such as the FTS, are typically equipped with video equipment which 
conform to the National Television System Committee (NTSC) standard used in television 
broadcasting. When NTSC video is displayed on a 20-inch monitor and viewed at a standard 
computer-screen viewing distance, each pixel displayed subtends about four minutes of angle 
of the operator’s visual field. This corresponds to an image resolution about four times worse 
than that of the average human eye [Ref. 24]. Consequently, a significant amount of visual 
information of a remote scene may be absent from such an NTSC display. 

Although a U.S. standard HDTV display format has yet to be finalized, it will likely 
have the same approximate pixel count as Japanese and European HDTV display formats. 
Such an HDTV display format will increase visual resolution by a factor of two over the 
NTSC standard and will come to within a factor of two of the human eye’s imaging 
capabilities. Thus, it is evident that the replacement of NTSC with HDTV visual interfaces 
will greatly increase the amount of visual data presented to the TOR operator. HDTV TOR 
interfaces will likely be one of the key enabling technologies necessaiy for achieving TOR 
Telepresence Control (TOR TC). 

In TOR TC, the remote human operator should perceive the robot and its environment 
near the inherent resolution and bandwidth limits of the human senses. For example, a 
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large HDTV screen filling the operator’s entire visual field can convey a more precise and 
inclusive image of the TOR environment than a small, low-resolution TV displaying the 
limited field of a view (FOV) of a video camera. The significance of high-resolution imageiy 
is easily overlooked but not easily quantified. Researchers at the Armstrong Aerospace 
Medical Research Laboratory, Wright-Patterson Air Force Base, Ohio, are now trying to 
determine the optimal level of visual sensory input data and the best geometrical 
presentation format for human teleoperators (two-dimensional flat screen display, two- 
dimensional binocular display, three-dimensional flat screen display, etc.) [Ref. 25]. In 
military programs to develop remotely piloted vehicles (RPVs), it has been found that remote 
pilots using low-resolution narrow FOV imagery have a much higher incidence of pilot error 
than real pilots. Many remote pilot errors may result from limitations of the operator 
interface. Likewise, TORTC may significantly improve TOR operator awareness and 
productivity. 

Many potentially revolutionary computer and entertainment interfaces are under 
development for commercial and military markets by a wide array of companies. This 
collection of technologies also holds tremendous promise for TOR TC. However, we note that 
large HDTV displays may not be desirable for extraterrestrial TOR command and control 
centers because of the weight and bulk of large Cathode Ray Tubes (CRTs). Lightweight flat- 
panel HDTV displays, such as those under development in Japan and the United States, 
would be preferred for TOR command and control centers at lunar or Martian bases. 

Another new display technology is especially well-suited for extraterrestrial 
applications because of its light weight and small size. Goggle-like devices called “eye 
phones” are under development by several companies (e.g., Sense8 Inc., Stereographies 
Reflection Technology Inc.) that entirely replace TV or computer screens with a set of sealed 
eyeglasses [Refs. 26, 27]. The first eye phones were developed by Ivan Sutherland in the 
1960s but were heavy and bulky because they used CRT displays. More recent eye phone 
systems are much lighter because of the lightweight LCD displays and compact integrated 
circuitry used. 

Eye phones can potentially display images with extremely wide FOVs because of the 
proximity of the focal plane to the eye. They also naturally display stereoscopic imagery and 
can be used to completely block out visual stimuli from the real world, thereby giving the 
user the impression of immersion in a virtual environment. According to International 
Telepresence, a maker of RPVs, stereo eye phones are increasingly being used in RPV control 
systems [Ref. 28]. However, their use for RPV control has presented interesting new 
problems for researchers. For example, operators of ground RPVs sometimes suffer from 
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motion sickness when the RPV is driven at high speeds. Apparently the lack of kinesthetic 
feedback has been determined to make even some tough U.S. Marines nauseous [Ref. 29], 
Nevertheless, the remarkable capability eye phones provide may prove especially useful for 
the virtual environment or cyberspace TOR control interfaces described below. 

Researchers at Xerox’s Palo Alto Research Center are developing new types of three- 
dimensional computer user interfaces which take the now-familiar two-dimensional 
Macintosh window interface one step further [Ref. 30]. Three-dimensional graphical user 
interfaces could also be useful in increasing TOR productivity and perhaps achieving three- 
dimensional TOC TC. 

True three-dimensional displays are also under development at Texas Instruments’ 

(TI) Computer Systems Laboratory in Dallas and at MIT’s Media Lab in Cambridge, 
Massachusetts [Ref. 31]. TI is now developing system prototypes of its Omniview system for 
several military clients and expects to field operational systems by 1992. The MIT system 
can present more realistic 3-D imagery but is still in the research phase. 

TI's Omniview system creates a 3-D display volume by rotating a disk around an axis 
perpendicular to the disk’s center in its plane at 600 rpm. A low-power pulse laser beam 
modulated at 10 kHz illuminates the disk. Because of the system’s high interlace rate, the 
sequential pattern of points of light created in the display volume is seen by the human eye 
as a solid three-dimensional display image. Red, green, and blue lasers can be mixed to 
produce a full-color 3-D display that can be viewed at any viewing angle. The one drawback 
of the Omniview system thus far is that it can only display transparent objects (nested 
objects are not occluded). 

In the MIT system, real-time holographic images are created from CAD data files by 
using a massively parallel supercomputer — the Connection Machine built by Thinking 
Machines Inc. Supercomputer speeds are required because of the highly computer-intensive 
graphical imaging algorithms needed to account for object occlusion and viewer parallax 
effects. Even with supercomputer speeds, this system can only create small images (about 
1.5 inches on a side) and they can only be viewed three-dimensionally from a small number of 
vantage points. 

Although many of the display technologies described above are still in the development 
phase, they hold promise for powerful new TOR TC interfaces and bring telepresence closer 
to the ideal of actually being there, which is why commercial companies are now vigorously 
developing these systems. NASA needs to follow these commercial and military 
developments and integrate them into current robotics programs. 


-31- 


The second key set of technologies required for TOR TC interfaces relates to the tactile 
interface between controller and robot. Advanced remote kinesthetic control systems have 
recently reached the stage where remote operators can be provided with a “feel” of the objects 
being manipulated by a remote robot. Control signals sent back to the manipulators can 
include the effects of synthetic forces experienced by the remote operator. High-fidelity 
kinesthetic sensing is another important enabling technology for achieving TOR TC. 

Finally, a third type of advanced technology could also greatly enhance the 
productivity and realism of TOR TC— Speech Synthesis and Recognition (SSR). If a TOR 
controller could verbally command a robot, other more cumbersome and slower control 
interfaces could be eliminated (keyboards, trackballs, etc.). SSR research has been ongoing 
for some time, but, to date, few if any real practical applications have emerged. However, it 
now appears that SSR technology will soon be emerging from the research lab and going into 
commercial products such as computers and VCRs. For example, Matsushita Corp. will 
introduce a VCR that can be programmed by voice command by 1991. Motorola, Intel, and 
AT&T have all recently introduced “Media Engine” chips destined for use in personal 
computers and work stations, which will allow users to store and retrieve audio, visual, and 
text data from a computer [Refs. 32, 33, 34]. Recently, AT&T and several other 
telecommunication equipment makers have introduced telephone message systems which 
recognize and interpret simple voice messages [Ref. 35]. IBM has also recently introduced a 
relatively low-cost personal computer, which after some training can be run entirely by voice 
command. This IBM PC runs several standard software programs and has a vocabulary of 
approximately 7,000 words [Ref. 36]. New software products will undoubtedly be created 
that will provide a verbal command interface with software application packages. 

None of the submissions that scored highly in the review process specifically propose 
new technologies such as those described above, although several other submissions, which 
did not score as highly, refer to telepresence, HDTV, IMAX or other high-resolution display 
technologies for SEI robotic and remote monitoring systems. 1 The following submission, 
however, scored highly because it discusses how SEI telepresence systems could be used in 
innovative ways on space stations, to repair and retrieve satellites, and to support 
commercial research ventures in LEO and on the Moon. 

1 IMAX is the name for the largest motion picture film format available commercially. Each 
IMAX film frame contains roughly eight times the number of pixels (film grains) as a 35mm film frame. 
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• Telepresence and Commercial Mission Objectives (#100827) 

The submission proposes extensive use of telepresence technologies in space by NASA 
and argues that with the many competing commercial ventures in space in the time frame of 
Project Outreach (2019), private corporations will insist upon using TOR TC to reduce launch 
costs, astronaut time, and manpower. This is illustrated by the following example: TORs on 
a space station could provide an engineer on Earth with a simulation of the space station 
laboratory and allow the engineer to observe and manipulate experiments almost as if 
present at the station. It is proposed that laboratories in space be engineered, by 2019, to 
use telerobots and permit researchers on Earth to conduct experiments in space that are 
expected to be of commercial value. The proposer maintains that relatively small robots, 

< 0.3 meters in diameter, could provide a wide range of services under teleoperator control. 

The use of telerobots to support a wide range of commercially motivated experiments, 
as well as many others, is reasonable now and will certainly be extensive by 2019. It is likely 
that many types of robots will be required. It is proposed that robots could be “ganged” 
together to do larger tasks. This is not possible now but could be in the future. It is certain 
that space facilities must be intentionally designed to accommodate such robots just as 
laboratories and shops are designed to accommodate equipment on Earth. Safety issues 
dealing with astronauts operating around robots were not discussed. 

Virtual Environments, Artificial Realities, and Cyberspace 

The integration of the advanced display, audio, and kinesthetic force-reflection 
technologies described above with new proprioceptive motion sensors and computer 
simulation techniques will bring about an entirely new type of entertainment and computer 
interface. This integration is now the focus of many academic and NASA researchers, U.S. 
and Japanese corporations, and the Japanese Ministry of International Trade and Industry 
(MITI) [Refs. 24, 27, 37]. The products of this integration effort have been called artificial 
realities, virtual environments, and cyberspace by various authors and researchers. 2 

Cyberspace has three components: sensory construct, behavior simulation, and user 
interaction. Realistic imagery, tactile forces, and sound present the user with a single 
inclusive sensory construct. This sensory construct is made to behave exactly as real objects 
would by computer simulation. Finally, the user interacts with the sensory construct much 
as he or she would in the real three-dimensional world: by moving, pointing, and picking 
things up, by talking, and by observing from many different angles. Cyberspace can be 

2 The earliest discussion of virtual environments or artificial realities appears to be in The Joy 
Makers by James Gunn in 1961 [Ref. 38]. 
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thought of as an artificial reality which by computer simulation obeys the laws of physics and 
the commands of the user. 3 Vitual environment software products are being developed by a 
number of firms. Autodesk, a maker of CAD/CAM software, is designing an artificial reality 
operating system which will enable other software developers to create their own virtual 
environments for entertainment, physical fitness, or education [Ref. 40]. 

Proprioceptive motion sensors that permit interaction with displayed or virtual objects 
are under development in many companies and research organizations (e.g., VPL Research 
Inc., MIT Artificial Intelligence Laboratory, Columbia University, Sense8 Inc., Mattel Co. 
[Refs. 24, 27, 30, 37]). The operator wears a “power or data glove” containing proprioceptive 
motion sensors. As the operator waves his gloved hand through space, he sees an image of 
his hand move through the space displayed on his screen or eye phones. In this way the 
operator grasps, moves, and rotates virtual objects. The application of these technologies to 
video games is obvious. Mattel's Power Glove has in fact recently been licensed to Nintendo 
for this use. Nintendo has also developed other motion sensors for video game interfaces. 

One of them is called “Power Pod,” which uses switches embedded in a plastic floor mat to 
signal a player's motions or decisions. 

These technologies also permit development of TOR Cyberspace Interfaces (CSIs), in 
which objects imaged by the TOR are presented to the TOR operator for manipulation. The 
TOR operator perceives his own arms to be those of the TOR, and the virtual objects in his 
visual field to be the real structures grasped and manipulated by the TOR manipulator arms. 
Increasingly sophisticated CSIs will no doubt be developed and used in the coming decades. 
Besides their clear utility for TOR control, CSIs could also be used by NASA in its public 


3 Cyberspace is a concept that encompasses virtual reality. Within cyberspace one can view a 
simulation of a past or present reality, such as depicted in the novel Neuromancer [Ref. 39], the movie 
Brainstorm, or the holodeck on Star Trek — The New Generation, However, the novel Neuromancer, or 
to a lesser extent the Walt Disney movie Tron , depicts a broader manifestation of virtual reality in 
which physical, mathematical, and data structures of a computational system are made as visceral and 
observable as the real world. Within this visceral presentation of computational abstractions things 
like encryption algorithms might become visible as walls or moats. Algorithms might be viewed as 
carnivorous animals that consume one type of data “animal,” and convert them into another form, new 
“organisms,” or waste. Applied mathematicians now explore a small segment of this abstract virtual 
reality by using animated computer graphics to study new forms in topology or very complex and non- 
con tinuous transformations such as Mandelbrot sets. 

A conceptional domain that is yet to be explored is the use of the broader form of “virtual reality” 
to study the control of robots in simple and complex environments and the strategies for obtaining 
progressively higher forms of autonomy. Cyberspace, a simple interactive version of which can be 
provided now, should be explored as a means of simplifying the control of robots in complex situations 
and under time delay. 
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relations efforts and allow the public to participate vicariously in a new and exciting aspect of 
the robotic exploration of space. 

• Creation of a Virtual Environment for Teleoperation (#101317) 

The concept of cyberspace is much broader than that of teleoperation. The operator 
interface of a teleoperation system presents the user, as closely as possible, with a 
representation of the environment of the robot at the time the signal is transmitted from the 
robot to the teleoperation center. Cyberspace interfaces can include such a slice of “reality” 
but can also extrapolate to future times by simulating the probable time evolution of the 
user’s virtual environment. 

CSIs can also present completely synthetic simulations, thereby extending for the 
operator the types of robot operations to be considered. CSI provides great safety for the 
remote operators, especially if other robots can repair a disabled robot. As the 
communication time between the robot and operator increases, CSIs will become more useful 
than simple teleoperation. Low-level CSIs, such as flight simulators or even video games, 
are now possible and extremely useful. The submission provides no technical details on how 
CSIs can be implemented and in particular how time delays in interplanetary teleoperation 
can be overcome using specific CSI simulation software. Nevertheless, the opportunities for 
innovation suggested by this submission are truly immense, and NASA will likely be hard- 
pressed in the future to make full use of the CSIs that will become commercially available. 
Communications links over vast distances and modeling non-terrestrial environments will be 
among the uses of CSIs NASA will want to consider. 

• Computer-Simulated Teleoperation (#100336) 

Computer-Simulated Teleoperation (CST) is a restricted form of a CSI. A slowly 
moving rover on a distant planet, say Mars, continually makes pictures of its proposed route 
and sends them to Earth. The pictures take X minutes to get to Earth. The CSI system on 
Earth presents the operator on Earth with computer extrapolations of where the rover will 
be on the basis of pictures taken at -X minutes earlier. The operator sends the commands 
based on the extrapolated pictures to the rover. At +X minutes the rover receives the control 
inputs from Earth that are based on extrapolations from -X minute observations with its 
observations and compares them with its observations at +X minutes. If there is reasonable 
similarity, then the commands are implemented. Such a system can be extensively tested on 
Earth. 

This version of a CSI can increase human safety if people do not have to work around 
the device. The average rate of travel of a sequence of machines, a convoy, can be much 
faster than one machine operated through a time delay. The technology is reasonably 
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available for simple, structured environments. It is beyond the state of the art for unknown 
environments but is worthy of development. 

• On-Orbit Assembly, Servicing, and/or Maintenance Incorporating 
Teleoperations and Control-Structure-Interaction Technologies 
(#10081) 

This particular abstract poses research on a small but significant portion of the 
nonlinear controls problem. It is not highly innovative; many others have suggested similar 
motivations. It lacks details to provide insights as to unusual technical advances the group 
might use. However, nonlinear control systems will allow critical tasks to be done in space 
that are simply not possibly otherwise. 

Assembly and maintenance of large, low-mass, extended structures such as space 
stations, large arrays, or incomplete structures will require robots and manipulators that are 
driven by nonlinear control systems. The dynamics can be nonlinear and very complex. 

Thus, nonlinear control systems will be required to ensure safety of the structure during 
assembly and maintenance. 

Development of such systems will require much work on both simulations and real 
tasks. Great advances have been made in the past 20 years in the development of nonlinear 
systems. However, it is still an area of fertile research and will require continuing 
development of theoretical understanding, software, and computing techniques, as well as 
taking advantage of emerging hardware such as parallel computers and neural networks. 
Likely, there will always be a boundary of unsolved problems and a tradeoff between various 
engineering approaches, mathematical understanding, and what cannot be done. Use of 
proven and trustworthy nonlinear control techniques could lead to mqjor savings in 
operational facilities in space. The technique development will be modest in costs compared 
to flight programs and the monies that might be saved. 

TOR COMMAND, CONTROL, AND COMMUNICATIONS (C3) SYSTEMS 

SEI TORs that will be used in LEO, on the Moon, or on Mars will be controlled by 
human operators located in the same area or possibly on a nearby planet such as Earth. For 
many tasks envisioned as part of SEI, several robots working in close proximity may be 
required. For example, construction of huge spacecraft such as MTVs in LEO or on the Moon 
may require many TORs working together like a construction crew. In such cases, TOR 
activities will have to be carefully planned and coordinated. TOR work difficulties may occur 
and real time coordination or retasking may be required. As more TORs are deployed on 
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space construction projects, complex task schedules will have to be developed and 
promulgated to the appropriate controllers. 

The additional TOR tasking, planning, coordination, and control activities that will 
have to be handled in complex space or lunar construction activities suggest that TOR 
command, control, and communications (C3) centers will be needed to manage such 
operations. A TOR C3 center would be composed of TOR control stations, coordination 
managers, logistics managers, planners, and taskers. The requirement for such a facility is 
suggested by analogy with the types of C3 centers that have been developed for complex 
military operations. Real-time and nonreal-time management of complex systems like TORs, 
especially when several are working in the same vicinity, will likely require many 
unspecified but important group interactions among TOR controllers, taskers, and logistics 
personnel. Such a TOR C3 center is proposed in the following submission. 

• The Robotic Workshop (#100337) 

This submission proposes a workshop containing many different types of small TORs 
with many different capabilities— -different speeds, strengths, precision, etc. Each TOR 
would be provided power, computing, control, and communications from a single source in 
the workshop. The robots would be in a single room or work area that could be fully observed 
by remote operators. The remote operators would be placed in a single room as well to 
closely cooperate in coordinating the control of several different robots at any one time. The 
workshop control center (TOR C3 center) could be in a developmental lab on Earth, in LEO, 
or on the Moon. 

The concept is very good in that it can be implemented quickly and inexpensively for 
an Earth demonstration, and it would rapidly develop understanding of how robots can be 
coordinated to do complex tasks both with and without time delay. The workshop might be 
translated quickly to a LEO space facility and thereby greatly increase the productivity of a 
laboratory inside or outside the facility by allowing controlled access by the earthbound team. 

Development of supervised autonomy and autonomous robots is not part of this 
workshop concept. The workshop could be used as a environment for development and 
demonstration of supervisory control and checking out of autonomous robots. As mentioned 
above, the fundamental concept is not new and can be rapidly developed. 

SEI ROBOT MISSION PLANNING 

As the SEI program proceeds over the next quarter century, SEI operations in space 
will increase in scope and complexity. As SEI robots become increasingly capable, they will 
be given greater responsibilities and more complex tasks to perform. Succeeding generations 
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of SEI robots will depend upon and exploit data collected from previous SEI and NASA 
missions. Data collection requirements on early missions should therefore be carefully 
determined with later SEI mission data needs in mind. In this way, synergies can be found 
between early SEI data collection efforts and later exploratory or colonization missions. 

A second aspect of mission planning that also must be addressed has to do with 
capturing essential design information of SEI systems right at the start of the design stage 
by using integrated CAD/CAE tools. CAD/CAE data files will be essential in later robotic 
repair or assembly operations in space, as explained below. If CAD/CAE data are archived 
and configuration controlled, a significant increase in the productivity of SEI robots could be 
realized. 

Remote Sensing, Route Planning, and Navigation 

• Image Processing by Lunar Rovers (#101067) 

This submission is an innovative proposal that uses image processing techniques to 
reduce the size, complexity, and cost of lunar rover exploration robots. The rover operating 
principles that would be employed will be discussed later in this section. This submission 
also demonstrates how data collected by early SEI probes (in this case the Lunar Observer 
(LO)) can be used for extensive and detailed mission planning for later lunar exploration 
missions (in this case by lunar rovers). This submission suggests an unprecedented data 
collection effort using remote-sensing techniques to construct high-resolution synthetic 
imagery (.1 m resolution) and maps of the lunar surface. 

Although the submission suggests that LO imagery be transmitted to Earth, processed 
in real time, and then transmitted to the lunar rover, this type of real-time communications 
connectivity is not necessarily required. With the absence of an atmosphere to erode or move 
the lunar regolith, the lunar surface has been and will remain unchanged for centuries. LO 
imagery can be collected, buffered on the satellite, and transmitted back to Earth. On Earth, 
supercomputers would have ample time to develop high-quality synthetic ground-level 
imagery using sophisticated image processing and translation algorithms. Synthetic imagery 
could then be loaded into the lunar rover’s memory in nonreal-time by radio link or by 
ferrying high-capacity memory cards or disks from Earth to the Moon. 

High -resolution remote sensing of the Moon performed on the scale suggested (.1 m) 
will also provide an unprecedented geological record. Advanced high-density data storage 
systems based on first-generation optical storage techniques now available will be capable of 
preserving this record for centuries. The lunar database could not only be used by NASA 
mission planners, but also by scientists and private commercial ventures formed to prospect 
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for lunar resources. In addition, this high-resolution database can naturally be used to 
create cyberspace software for the general public. The new telepresence and CSI 
technologies described earlier could be used for educational and entertainment purposes by 
those interested in space exploration and experiencing first-hand the lunar surface. 

Although the submission does not identify specific sensors to be used, optical 
photography is implied, since the current LO baseline system includes an optical camera 
capable of .5-1.0 m per pixel resolution [Ref. 41]. Optical imaging at .1 m resolution is clearly 
possible by upgrading the LO camera system. However, because ground-level imagery must 
be synthesized from high-altitude imagery, LO image resolution may have to be greater than 
.1 m. The question requires detailed study beyond the scope of this Note. 

There is a significant drawback to employing only high-resolution optical photography. 
The permanently dark craters at the lunar poles could not be imaged, so rovers could not be 
provided for ground-level synthetic imagery for navigating these areas. An alternative 
approach is to employ active high -resolution imaging sensors such as Synthetic Aperture 
Radar (SAR). In addition, because the Moon has such a tenuous atmosphere, EHF and 
millimeter wave SARs could generate high-resolution imagery. To achieve the resolutions 
discussed, the current state of the art in these sensor technologies would have to advance. 
However, such developments would have many spinoffs for other remote sensing 
applications, including advanced tactical military imaging systems, environment monitoring 
sensors, and arms control verification systems. In addition, SAR data will provide surface 
and subsurface geological data which would be useful for resource prospecting in its own 
right. 

Another approach to generating high-resolution optical imagery may also be feasible. 
Instead of employing an upgraded LO spacecraft in a 100 km circular orbit, smaller 
“lightsats” carrying a smaller array of instruments could be deployed in extreme low-altitude 
(10 km) orbits. Lightsats could simultaneously or subsequentially image the same surface 
swath from adjacent orbital planes. Distance measurements between a lightsat pair and the 
surface swath could be taken using laser range finders or by using precise satellite ephemeris 
data, thereby providing accurate cartography as well as images. Recent advances have led to 
a reduction in the size of space sensors and microelectronics. These advances have led to a 
variety of innovative lightsat concepts for commercial and military applications in various 
Earth orbits [Ref. 5]. NASA and JPL should examine these emerging technologies and see 
how they can be used to fulfill SEI mission objectives. 
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Remote Sensing and Lunar Base Construction 

Submission #101067, discussed above, has important mission-planning implications for 
other aspects of SEI. The first settlements on the Moon and eventual lunar bases may be 
completely or partially assembled by robots. Robots will be needed to excavate, move, and 
smooth lunar regolith for roads, habitats, and power sources. Much of this activity will have 
to be performed in an unstructured robot work environment if mission planning for these 
activities relies only upon medium resolution imagery. In this case, TORs, which may have 
to be controlled from Earth in slow feedback loops, would have to be used. On the other 
hand, if the location for the lunar base could be imaged at high resolution, more detailed 
mission planning for base construction could be done on Earth, and it therefore may be 
possible to employ advanced STRs such as those suggested in submission #100378 for some 
assembly and construction activities. Such STRs, if appropriately programmed and equipped 
with limited autonomous small-obstacle avoidance capability, may be able to independently 
carry out lunar construction activities without direct control from Earth or lunar transfer 
vehicles (LTVs). 

Computer Aided Design, Database Management, and Expert Systems 

As discussed by many authors [Refs. 1, 4, 16, 17, 42], an essential aspect of mission 
planning for SEI robotic assembly and repair activities is provision of detailed CAD/CAE 
data for SEI robots. CAD/CAE data files can be applied in robotic repair or assembly 
operations in space as explained below. If this is done, a synergistic increase in the 
productivity of SEI robots could be obtained. While no high-scoring submissions directly 
advocated or addressed the utilization of CAD/CAE data by SEI robots, one submission 
dovetails nicely with these concepts. 

Submission #100345 proposes the development by NASA of a modular robotic control 
architecture that can support a range of robot applications and integrated sensor systems. 
This modular design would be hierarchical, supporting, among other things, various control 
levels ranging from high-level mission and task planning down to macros (sets of specific 
robotic arm motion commands) and low-level primitive robotic and individual servo 
commands. This ambitious venture is only sketched out in the most general terms in the 
submission, and no backup is provided. Nevertheless, it suggests what might be possible if 
such a modular control architecture were available. Archived databases of sensory data from 
CAD/CAE designs, and later from robots and probes, could be downloaded into other systems 
and appropriately synthesized and filtered to provide deterministic programming 
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instructions for lower-level robots such as STRs. This concept requires much further study 
and refinement, but it may enable development of robot teams in which limited forms of 
shared autonomy could be realized in a cost-effective manner. 

SEI EXPLORATION ROBOTS 

A number of exploration robots have been proposed in various Project Outreach 
submissions. They can be grouped according to the propulsion/locomotion systems employed. 
Extensive research over the years has been conducted regarding the capabilities and 
limitations of various robot exploration concepts. The robots with the greatest range and 
payload carrying capacity are wheeled rovers. However, wheeled rovers can only cover 
unobstructed level terrain. Walking rovers cover territory at a much slower rate and carry 
less payload but can traverse more difficult terrain. Crawling rovers can penetrate more 
inhospitable terrain and potentially burrow underground, but carry only small payloads and 
cover small areas. Lightweight hopping robots can cover somewhat larger areas but also 
carry only small payloads. Small “all terrain” robots must have power and communications 
capabilities; power and communication tethers could connect these smaller systems to a 
mother rover and a local power cart. Alternatively, small robots could be operated 
intermittently using solar power and could communicate with a mother rover via radio link. 

Still another class of exploration robots is ballistic probes, which are well suited for 
subsurface sampling and analysis. They could be fired from orbit or from a gun mounted on 
a rover. Ballistic probes could be useful for remote prospecting operations in steep craters, 
canyons, or mountain ranges. Ballistic probes would be equipped with radio transmitters for 
communication with passing satellites or nearby rovers. 

The smaller the robot and the simpler its locomotion system, the simpler its guidance 
and control can be. Furthermore, if the robot is expendable, it could be completely unguided, 
with all of its payload devoted to sensor or power functions. Such smaller robots could be 
carried by a more sophisticated TOR rover, which would explore a wide region until it 
reached an interesting obstructed area. There it could release smaller hopping robots which 
could jump into and explore craters or canyons. Such unguided robots may be able randomly 
to explore difficult obstructed terrain at low cost relative to more sophisticated alternatives. 
Such a diverse team of robots would be useful in many different types of exploration missions 
because of their flexibility. 

• Competition for Design of Exploratory Robots (#101321) 

This submission proposes that NASA hold an open market competition for the design 
of exploration robots, which could possibly be sponsored by private corporations (perhaps 
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some form of tax writeoff or corporate advertising would be permitted to defray corporate 
costs). NASA would specify general robot characteristics, telemetry interfaces, and power 
supply. The robot could be capable of autonomous operation or be remotely controlled 
through teleoperation. Qualifying entries would compete in an “Exploration Competition” on 
a suitable piece of terrain on Earth. The top two contestants would be guaranteed a berth on 
the first manned Mars landing, where they would be used by the exploration team. 

This submission is innovative and could provide a new impetus to academic and 
corporate researchers to develop new robotics technologies. It would also provide a small but 
perhaps significant way to side-step the difficult and often criticized government 
procurement process. Because of the publicity such a competition would engender, interest 
in SEI would also be promoted among the general public. For example, corporate sponsors 
could use their robots in advertising campaigns, and robots adorned with corporate logos 
would been seen on living-room HDTVs, moving about on the Martian surface. 

Wheeled Rovers 

• Computer Simulated Teleoperation (#100336) 

This submission was discussed above in the context of virtual reality and its 
applicability to operating rovers on Mars. A remote operator on Earth, or anyplace 
sufficiently far away to cause a delay in communications, will generally issue robot control 
commands on the basis of interactions with a computer projection of where the robot should 
be, based on post-event rather than real-time data. The submission proposes that 
experiments be conducted in Arroyo Seco, on the grounds of the Jet Propulsion Laboratory, 
using wheeled robots and communications schemes that introduce a time delay. Wheeled 
robots and adjustable time-delay equipment are readily available. 

Development of virtual-reality software for vehicle control should be initiated. The 
main goals should be to minimize the information that must be transmitted and received 
from the robot and to determine how the robot can compare, at the least computational 
expense, its real-time environment with the simulated environment with which the operator 
is interacting. There is nothing particularly special about the use of wheels with respect to 
the control modality, except that a wheeled robot may be readily available at JPL. 

• Image Processing by the Lunar Rover (#101067) 

The NASA 90-Day Study proposed placing a mapping satellite in orbit about the Moon 
that would take “moderate” resolution photographs of the surface. The proposers presume 
that moderate resolution is significantly greater than 0.1 m. We conjecture that if high- 
resolution remote sensors were employed, operation of unmanned rovers on the lunar surface 
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could be considerably simplified. Otherwise, the rover would have to have a high level of 
autonomy to navigate on the insufficiently resolved surface or a teleoperation would be 
required. The proposers view autonomous operation as unrealistic and teleoperation as too 
slow. 

The proposed solution requires four elements. The lunar orbiting satellite would have 
a 0.1 meter resolution mode for imaging areas to be explored by rovers. These images would 
be transmitted to Earth. Large computers on Earth would combine orbiter images taken 
from different perspectives and would synthesize from them highly accurate three- 
dimensional maps with ground-level reference imagery for use by the rover. Local maps and 
at-request synthetic imagery would be transmitted back to the rover to assist it in 
navigation, hazard detection, and other functions such as sample collection. The rover would 
also take high-resolution images from ground level and compare them with synthetic images 
to resolve any navigational problems which may be encountered. Resolved images would be 
transmitted back to Earth and could also be used to generate “virtual reality” 
representations of the lunar surface for use by scientists, engineers, and the general public. 

By performing the complex image processing on Earth and transmitting processed 
images to the Moon at high power levels, the computational and power requirements on the 
rover can be easily met. The rover could be smaller because it could be designed to navigate 
around even small objects (< 0.1 meters). Teleoperation could be minimized because the 
rover would operate from an accurate map of the territory and the rover could operate at a 
relatively high speed (> 1 meter/sec). 

These claims all seem reasonable and are based on technologies in use on Earth. The 
technique would be extendable to other planets and quite useful when the emphasis is on 
detailed exploration of a particular region in which optical photography is possible. Active 
imaging systems, such as synthetic aperture radar, may be needed to map dark regions such 
as the lunar craters near the poles of the Moon, where some scientists expect water-ice will 
be found, and on Titan where the surface is obscured by clouds. Costs may be driven by 
telemetry and remote-image sensor complexity. The approach should be carefully examined. 

Walking, Hopping, and Crawling Robots 

• Wheeled Articulating Rover Propulsion Methods (WARPM) (#100815) 

A rover is proposed that has “legs" having “powered wheels" as feet. Walking motion 
would be used in rough terrain and rolling motion in smooth terrain. Wheeled travel would 
allow the robot to efficiently travel long distances over smooth terrain. However, there is 
concern that the complex wheel and power system will be subject to a wider range of failures 
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and significantly degrade the walking ability of a legged robot. Development costs are likely 
to be higher with two modes of travel. Two separate vehicles, one wheeled and one a walker, 
that can carry one another might also be considered. The concept has been proposed many 
times before. It should be considered in various SEI mission niches after a careful review of 
the available literature. 

• Solar Powered Cricket (#100377) 

Mechanical crickets of less than 10 kg mass would be provided that are equipped with 
solar cells for power, a mechanical arm for hopping, instrumentation for data-gathering, and 
telemetry for local transmission of data. Several crickets would be deployed from an 
unmanned lander for initial exploration of a particular area. They would periodically hop 
around the landing area and over a period of time gather detailed statistical data. The 
crickets would be expendable and equipped with minimal guidance mechanisms to recognize 
and clear major obstacles and with mechanical means of reorienting themselves after a 
landing. 

This interesting concept should be explored. The approach offers a low-power method 
to explore a small region on the Moon or Mars. Advances in micro-mechanical and micro- 
chemical devices would enable a swarm of rugged crickets to gather a wealth of local data. 
The cost could be moderate and a swarm of crickets would be more reliable than a single 
rover. 

• Crawling Rover/Manipulator Project (#101325) 

A multibody vehicle is proposed that is composed of “leg pairs” Each leg pair consists 
of a payload box, two lateral, rigid legs, and an actuation mechanism, termed a Stewart’s 
platform, connecting one payload box to the next. The Stewart’s platform is an octahedral 
cell with six variable-length actuators that move one payload box (rotation, differential 
length) with respect to the preceding and following payload boxes to produce forward and 
turning motion in the style of a caterpillar. The combined units could serve as either a 
robotic arm or a mobility device. Thus, these types of units might find use in facility 
construction and surface mobility, a unique combination. 

A detailed technical paper and outline proposal for development of the computer 
program to enable coordinated redundant control was included. The submission received 
mixed reviews. It was considered a veiy safe but slow approach to surface mobility. The 
device would have little stored energy and that energy would be restricted to a few of the 
body segments at a time. That is a significant safety feature. Various payload boxes could 
support different functions (mechanical, sensors, etc.), and great redundancy and stability is 
inherent. The multibody vehicle offers a unique combination of hard and soft automation 



-44- 


and can be demonstrated in a laboratory. However, the advantages gained by crawling could 
be negated operationally by the considerable mechanical and control complexity. The concept 
should be further considered by SEI. There are significant terrestrial applications such as 
repair and inspection inside pipelines or sampling of very rough terrain. 

Ballistic Probes 

• Automated Subsurface Sampling by Coring Penetration (#100339) 

Several coring penetrators, described as optional exploration technology for a Mars 

Sample Return Mission (MSRM) defined by JPL, will be released from an orbiter and fall to 
Mars within a predetermined area. The various penetrators drive to different depths, all 
greater than 2 meters, and somehow provide their samples to an Automated Sampling and 
Collection System (ASCS). The ASCS is delivered to each penetrator by a rover that operates 
from its lander and sample-return vehicle. 

This technique has been studied for Mars, asteroids, and the Moon, and penetrators 
have been demonstrated on Earth. Such penetrators would increase the initial costs and 
complexity of a sample return mission. Inevitably the samples would be modified by the 
penetration and collection processes in ways that would be hard to characterize without 
reference tests that are impossible in first missions. The penetrators would land randomly 
inside a given area and, except for penetration depth, would not necessarily return the 
widest variety of samples. The technique should be reexamined by SEI for future missions 
but only in competition with other more controllable techniques that would also not be as 
prone to single-point failures, such as might occur with a single rover and single ASCS. The 
unmanned system is seen as safe for humans but of relatively low reliability and of average 
utility and innovativeness in comparison with other approaches. 

Diversity and Teamwork 

• The Lewis and Clark Expedition II (#100343) 

A general approach is sketched out for the use of three types of robots to conduct 
unmanned, remotely controlled exploration of specific traverses along the lunar surface. A 
transporter and base unit robot would take two different types of exploration robots (Lewis 
and Clark) to a particular site. The site, such as a crater wall, might be too rough for the 
base vehicle to traverse. Lewis and Clark, which are not described, would conduct separate 
surveys and provide backup to one another. 

The general concept is interesting, because each robot could be specialized and 
therefore possibly cheaper to build and deliver. The use of several types of robots may 
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increase system reliability. The rover could be generally similar to the Soviet Lunokhod 
deployed in the early 1970s. The basic architecture is not innovative. SEI should consider 
the use of different types of robots in exploration, construction, repair, and other functions as 
suggested in several of the submissions. 

TRANSITION TO AUTONOMOUS ROBOTS 

Although the concept of the autonomous robot is an old one, it is still far from being 
realized and may never truly be created by human beings. Robotics has a host of unsolved 
problems, which make it appear that machine autonomy will not be achieved in the 
foreseeable future. For example, robots cannot “see” and understand images like humans do. 
Nor can they perform complex mechanical tasks in an unstructured, unpredictable work 
environment. Perhaps the real crux of the matter is that robots, even if equipped with AI 
software programs or expert systems, are not capable of reproducing all the rich and varied 
cognitive decision-making processes of the typical human. Nevertheless, it is informative to 
speculate on what capabilities an autonomous robot must have. 

The ultimate autonomous robot should be able to take the place of a human and 
perform human physical or cognitive activities. A fully autonomous robot would be capable 
of performing any possible human task through an equivalent set of robotic tasks. Therefore, 
to define robotic tasks for an autonomous robot, one can use human capabilities and 
activities as a model. 

Humans have many capabilities: high fidelity stereoscopic vision, speech or language 
interpretation, stereoscopic hearing, sophisticated goal-oriented navigation, dexterous 
tactile-sensing hands, legs for locomotion, and an inner-ear balance sensor for stability. All 
these human systems are marvelously well integrated. 

Many simple human activities, such as searching for and picking up a coffee cup, 
require precise coordination of several of the human capabilities mentioned above. Let us 
delineate the fundamental physical and cognitive tasks in this simple case. 

Consider a person who, wanting a cup of coffee, looks for a coffee cup in her office. 

After searching through the cluttered office, she spots a cup handle behind a stack of books. 
She moves to the book shelf by walking around a desk chair, stopping at arm’s length from 
the cup. She directs her hand towards the cup, grips the cup handle, and picks it up without 
knocking over the stack of books. 

It’s a simple matter for a human to find a coffee cup. Now consider a robot directed to 
search for a coffee cup hidden in the same disordered office. What intermediate tasks would 
such a robot have to perform to retrieve the cup? These tasks are indeed fundamental to 
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autonomous robotic activity, including those which may be required to fulfill SEI mission 
objectives. They are: 

• Global sensing, navigation, and movement 

• Local sensing, navigation, and movement 

• Image processing and understanding 

• Expert systems and decisionmaking 

Global Sensing, Navigation, and Movement 

First, the robot would have to find the cup using its own array of imaging and perhaps 
other sensors. To conduct a thorough search, the robot would require a global model of the 
environment, in this case a model of the office. If it is assumed that the contents of the 
cluttered office change at random daily, the robot could not be programmed with an 
all-inclusive static model (a model of a structured environment). Instead, the robot would be 
programmed with a priori knowledge of only the simplest attributes of the room and its 
contents, such as the size of the room and a description of the furniture, books, and cups 
present. The robot vision system would have to distinguish between objects and select a 
coffee cup from the clutter. This simple task is a formidable and as yet unsolved problem in 
computer vision (the ability to discern specific objects in a “noisy” cluttered image). 

Nevertheless, assume the robot has correctly imaged and identified the coffee cup. 

The robot would then determine its relative location and move toward it to pick it up. 
Because there is a chair in the way, the robot must construct and follow a path to avoid the 
obstacle, stopping within an “arm’s length” of the cup. In other words, the robot must 
navigate within a global model of the environment constructed from static information 
“known” a priori and from new information acquired during global sensing. Then its 
locomotion systems must move it accurately to the desired location in the room. 

Local Sensing, Navigation, and Movement 

If it is assumed the robot has been successful at the global sensing, navigation, and 
movement tasks described above, it must still execute several additional maneuvers to 
retrieve the cup from behind the stack of books. It must sense the orientation of the coffee 
cup (e.g., the direction in which the cup handle is pointed) and the cup’s position relative to 
any obstacles (the stack of books). Once this local sensing task is performed, the robot must 
determine an appropriate trajectory path for its hand so it can grasp the cup handle, pick it 
up, and carry it away without knocking over the books. After the local navigation program 
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has computed an appropriate hand trajectory, a sequence of motor commands would be sent 
to the robot arm. The robot's controllers, Actuators, and motors would move the robot hand 
to within the vicinity of the cup. A similar set of commands would be sent to the robot's 
fingers to circle the cup handle. The arm would lift and carry away the coffee cup along the 
previously computed trajectory path. 

Image Processing and Understanding 

Global and local sensing capabilities of autonomous robots modeled after humans 
should be capable of processing visual information and extracting from it the position, 
identity, and orientation of objects in a cluttered and noisy environment. Indeed, human 
vision and associated image-understanding capabilities are understood in only the most 
general geometrical terms by current researchers. How the mind extracts object-oriented 
information from an image and understands the geometrical relationships between objects in 
a scene is still not understood. 

One should be careful to distinguish other forms of image processing that are more 
well developed, such as three-dimensional computer graphics, from the notion of computer 
vision. Three-dimensional computer graphics is, in fact, the inverse of the process we are 
concerned with here. Computer graphics engines take mathematically defined geometrical 
objects placed in specific orientations (for example, some objects occluding others), the 
position of the scene light source, and the viewer's position, and through a series of 
mathematical operations construct an image of a scene. Unfortunately, only rarely does this 
image-construction process have a unique inverse process. The more complex or noisy the 
image, the more inverse-image processes and visible-object sets correspond to the image at 
hand. Many common optical illusions, such as the famous etchings of Escher, are in fact 
based upon this non equivalence of images and visible-object sets. In order to understand 
images, the human mind uses many sophisticated and perhaps not always compatible image- 
processing algorithms. Many of these algorithms are still not understood, nor have they been 
translated into digital or photonic algorithms. 

No submissions were received in the areas of image processing or image 
understanding. 

Expert Systems and Decisionmaking 

For a robot to perform globally and sense, navigate, and move in autonomous fashion 
locally, it must continually compare data it receives from its sensors or other subsystems 
with its own world model and the command directives it has been instructed to follow. In 
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this data comparison process, the robot will eventually come to decision points where it must 
determine what to do next. These decisionmaking processes occur on many levels. When do 
I turn left to find my way to my objective? When can I safely stop tightening this nut and 
still be sure I have fastened the storage tank to the truss? When must I return to the power 
cart to recharge my batteries? 

Computer software systems, or expert systems, have been developed that can emulate 
these decisionmaking processes in some cases. Typical expert systems can make decisions on 
only a very limited but sometimes very detailed set of data or assumptions. In addition, 
these systems can sometimes be modified in real time when conflicting or new information is 
received. However, if unanticipated data or data not in the proper form or indirectly related 
to the data structures used in the expert system are received, the expert system may freeze 
up and not be able to incorporate the new data and make a decision. Because the fields of 
artificial intelligence and expert systems are still relatively new, it is not certain what 
capabilities will eventually be made available using expert systems. Several submissions 
were received in this area. All were fairly general endorsements of the technology and 
suggested what may be feasible if these systems are vigorously developed for SEI 
applications. 

• Self-adaptive, Scalable Real-time Control Architecture for Various 
Robotic Vehicles (#100342) 

This abstract, with no back-up paper, provides an extremely general description of 
some of the major goals of any effort to develop an autonomous robot. Such a system would 
“(a) accommodate a variety of sense-reason-act control models, (b) incorporate a ‘compare’ 
step in each model to dynamically modify its reasoning capability based on learned cases, (c) 
provide an exchange paradigm between the various models.” 

Computing systems with such capabilities would be extremely useful, especially if 
means can be provided to make sure the system is learning in a realistic manner and not 
developing capabilities to unexpectedly do harm to humans or critical elements of the 
mission. Unfortunately, the abstract does not provide pointers to the technical literature 
about technologies such as parallel processing, cooperative problem solving, or knowledge 
representations which are relevant to achieving autonomy. 

• Advanced Control Architecture to Support Various Missions, Robot 
Applications, and Integrated Advanced Sensor Systems (#100345) 

This abstract maintains that an object-oriented system of modular software can be 
developed that will provide progressively higher levels of autonomy as the development 
proceeds. The enabling technologies are stated to be the NASA/NBS Standard Reference 
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Model (NASREM) as developed for the Flight Telerobotic Servicer, the USAF Next 
Generation Controller Project for a Standard Open System Architecture Specification 
(SOSAS), the Product Data Exchange Specification (PDES), and the Product Definition Data 
Interface (PDDI). An extended paper with references and detailed logic was not provided. 

The architectures and specifications cited are rather general and are aimed at 
advanced manufacturing systems and robots to be operated within well-defined 
environments. Considerable real-world experience can be obtained when they come into 
widespread use. However, except for NASREM, NASA has little involvement in their 
development or use. They are not the focus of research in machine autonomy. NASA can 
certainly profit from examination of the systems, their operation, their application to systems 
designed to be supported by robots, and their aspects unique to space (e.g., control laws with 
variable gravity and long time delays). Practical experience on Earth becomes important as 
any level of manufacturing off Earth is planned. 

The generic concern is for NASA to systematically transfer terrestrial processing and 
manufacturing to the space environment and to support research on those aspects that are 
unique to space and therefore are not being developed on Earth. 

• Use of Next-generation Control Techniques for Robot/Machine Control 
Systems (#100348) 

The Air Force is heading the development of the next-generation control technology 
that is establishing the SOSAS for robotic and machine controllers. The market for devices 
using this system will be much larger than for any system currently specialized for space. 
Thus, NASA can learn much from the practical experience acquired by implementers and 
users of SOSAS. The challenges are: how to acquire knowledge of the practical experience 
that may be acquired worldwide (particularly from DoD), how to influence the development 
of devices, testing means, and data collection so as to transfer this experience to space 
systems, and how to recognize as early as possible the limitations of SOSAS for operations off 
the Earth. 

NASA must vigorously examine SOSAS. Practical knowledge of actual applications 
and demonstrations is a precious commodity. Perhaps NASA should sponsor design and 
demonstration studies of models and simulations of SEI operations using SOSAS standards 
and virtual reality simulations. NASA should certainly participate in SOSAS definition now 
and in the future. 

• Artificial Intelligence Systems for Space Applications (#100442) 

This submission, which contains an abstract and no backup paper, proposes that 

expert systems (ES) be applied to the control of various ground and space systems to reduce 
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manpower requirements by recognizing potential system faults, by identifying maintenance 
and repair requirements and procedures, by providing repair guidance for the operator and 
by training personnel for unfamiliar tasks. A list of current operating expert systems is 
given: Helix (helicopter operation), IFIP (fault isolation), and Sherlock (jet engine 
maintenance). Use of ES in planning and simulations is noted. 

Expert systems are now being developed for use on the space station and for future 
space experiments. They will be increasingly important in the future. A current 
fundamental limitation to their use is transferring data between program levels. For 
example, how can an expert system that provides a planning function receive information 
from an ES associated with supply of components and also from a lower system associated 
with monitoring maintenance needs of a unit of flight hardware? NASA must continuously 
study how ESs are applied in the economy at large as well as fund those ES applications that 
will reduce both ground and space personnel requirements. Expert systems are only one 
aspect of artificial intelligence (AI); thus, NASA should maintain a broad overview of the 
field. The aggressive use of AI in the development and operation of simulations will be 
especially useful in evolving AI and ESs from theory and commercial practice into the 
operation of the space program. 

Autonomous Navigation 

Development of true global navigational capabilities— especially for exploration 
rovers — for robots has been a subject of academic research for some time. Autonomous robots 
would have to possess such a capability and be able to formulate navigational cues quickly in 
order to traverse terrain or the space around a space station at high speed. 

• Three-Dimensional Reactive Navigation (#100333) 

This submission proposes development of an autonomous navigation program for 
three-dimensional movement that can also be reconfigured in real time in response to 
changes in the characteristics of the surrounding three-dimensional environment. The 
submission is accompanied by a backup paper whose subject is much more narrowly focused 
on three-dimensional obstacle avoidance techniques. Obstacles are modeled by a repulsive 
potential field, and the robot is guided along low-repulsive equipotential trajectories in the 
environment. The more ambitious claims of real-time reconfigurable reactive navigation are 
not substantiated in the backup article. Nevertheless, the submission touches on many 
interesting and important issues in autonomous navigation research. Such research needs to 
be funded to advance the state of the art of autonomous mobile robots, and especially that of 
exploratory robot rovers. 
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IV. CONCLUSIONS AND RECOMMENDATIONS 


In this section, we discuss our main recommendations to the Synthesis Group, then 
examine some important implications they may want to consider in the development and use 
of automated systems and robotics. 

A total of fifty-two submissions were received in the robotics area during Project 
Outreach. Most of the submissions were judged to be reasonable proposals, although there 
were a few submissions which seemed to fly in the face of both conventional wisdom and 
expert opinion. About half of the submissions (24) were judged to have high utility for SEI 
and were analyzed further by the robotics panel. 

Three types of robots were proposed in the high-scoring submissions: structured-task 
robots, teleoperated robots (like the FTS), and surface exploration robots. Several advanced 
TOR control interface technologies were proposed in the submissions. Many A&R concepts 
were presented by the submitters, but few specific technologies were suggested. There are 
many potential explanations for this. Proprietary submissions were not accepted. The time 
scale for Project Outreach was veiy compressed, leaving little time for a submitter to provide 
additional information. And finally, most submitters probably had to prepare their 
submissions on their own time. 

Review of the submissions and further research in A&R issues has led the Project 
Outreach A&R panel to submit the following recommendations to the Synthesis Group: 

• SEI robots, work environments, and systems should be systemically integrated. 

• Structured-task robots should be developed for SEI. 

• NASA should adapt and develop advanced TOR control interfaces which enable 
telepresence. 

• The architectural implications of using TOR telepresence control in SEI should be 
evaluated. 

• Data collection requirements for early SEI remote sensing missions should be 
reevaluated and harmonized with later SEI robotic mission requirements. 

• Tradeoff studies are needed to select optimum mobility and navigational 
subsystems for SEI surface exploration robots. Teams of complementary 
exploration robots should be considered in these tradeoff analyses. 

• Tradeoff studies are needed to determine the most cost-effective and productive 
development path towards autonomous robots. 
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• NASA’s evaluations of A&R effort for Space Station Freedom should be reviewed. 

Below we discuss these recommendations in more detail. 

INTEGRATE SEI ROBOTS, WORK ENVIRONMENTS, AND SYSTEMS 

Most human work environments can be unstructured, because humans can easily and 
rapidly adjust to unanticipated changes or events in their environment. Such human 
adaptability and flexibility result from our sophisticated and not completely understood 
planning, sensing, navigation, and movement capabilities. The current state of the art in 
robotics cannot provide systems that faithfully mimic these human capabilities, so SEI work 
environments in space and on the surface of the Moon or Mars must be carefully designed 
with the current limits of robotics in mind. SEI systems to be manipulated by robots should 
be designed so that robots can productively use their end-effectors. Further, all SEI robot 
end-effectors should be designed and manufactured to a limited set of end-effector design 
rules to enable different robots to use the same end-effectors for several different 
manipulation tasks. In particular, there are a number of FTS-like robots being developed by 
NASA or its contractors (submissions #100695, #100338, etc.). All these robots should be 
able to use the same end-effectors and these should be compatible with the hooks and scars 
being put in SSF and other SEI systems. 

A critical area that is not being considered in the United States but is under 
consideration in Japan is the development of space facilities that make extensive use of 
robots in assembly, maintenance, and repair. Robots are still viewed in the United States as 
gadgets or tools that are added to a structure to be constructed and maintained primarily by 
people. Extensive design exploration and demonstration efforts must be initiated to provide 
the United States with space and planetary systems that are primarily constructed, 
maintained, and repaired by robots. This critical theme was not explicitly mentioned by any 
of the submissions to the Automation and Robotics section of the RAND Outreach Panel. 

Perhaps the most important issue involved in systemically integrating SEI robots, 
work environments, and systems is capturing and maintaining configuration control over 
SEI system designs. Detailed engineering design data must be captured in a common digital 
format and made portable, so that it can be used by different system contractors during 
design and manufacturing and by robots in space during assembly and repair operations. 

The automated capture of SEI system design knowledge has been made possible with the 
advent of integrated CAD/CAM tools. Transportability of CAD/CAM files is also being 
improved with the introduction of commercial standards such as the emerging Electronic 
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Data Interchange Format (EDIF) CAD/CAM standard. NASA should monitor the 
development and use of CAD/CAM tools and standards in the semiconductor and other 
industries and adapt these increasingly powerful design tools to SEI systems and robots. 

DEVELOP STRUCTURED TASK ROBOTS FOR SEI 

The most productive robots on earth are STRs. They have transformed the Japanese 
auto and semiconductor industries. Now the Japanese install as many robots every year as 
exist in the entire United States industrial base [Ref. 2]. Equally productive robots will be 
needed for SEI if the President’s ambitious mission goals are to be met within the designated 
time frame and within future budget constraints. 

Much further research into the use of STRs in space is required. The work being done 
in this field (submission #100378) should be greatly expanded for SEI. Assembly tasks 
should be made easy and modular, enabling STRs to be used wherever feasible at SEI 
extraterrestrial operations nodes. 

Review of the submissions, and this panel’s research and inquiries, indicate that 
NASA A&R research and development activities may have been too tightly focused on 
expensive high-technology developments like the recently cancelled FTS. While the FTS 
program was a necessary and ambitious technology demonstration project, SEI funds should 
also be allocated towards development of STR work environments and STRs for specific SEI 
applications. The adoption of commercial STR technology, as represented by submission 
#100378, should continue and be expanded. These efforts can lead to highly productive and 
cost-effective space and lunar construction concepts and may generate commercial spinoffs of 
their own. Such activities can only help revive the moribund U.S. commercial robotics 
industry and also provide a natural upstream technology base for the eventual colonization 
and industrialization of the Moon. 

ADAPT AND DEVELOP ADVANCED TOR CONTROL INTERFACES 

Submissions #100695, #100338, #101469, #100827, #100336, #101317, and others 
propose that TORs be used for many SEI assembly, processing, repair, and exploration tasks. 
Because TORs can be remotely controlled by humans, they can operate in unstructured 
environments and are more flexible and adaptable than STRs. They also require much less 
complex real-time software than autonomous robots would need. As a consequence a variety 
of TORs have been developed for commercial and space applications whereas autonomous 
robots have yet to be realized. However, most TORs available today are cumbersome to 
operate and typically perform manipulation tasks much more slowly than humans. For 
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example, it is estimated that the FTS in its initial configuration will perform manipulation 
tasks in space at a significantly slower rate than a well-trained astronaut in an EVA 
spacesuit. The performance limitations of current TORs have therefore prompted 
researchers to develop new TOR control interfaces to improve TOR productivity. 

NASA researchers were some of the first to develop new and innovative display and 
interactive computer control technologies, such as “eye phones” and “power gloves,” which 
offer tremendous promise as TOR control interfaces. Now commercial companies, both in the 
United States and Japan, are racing to refine and extend these technologies for various 
consumer, scientific, and business products. In addition, HDTV, high-resolution flat panel 
displays, and new three-dimensional display volume systems are being developed. The 
leading edge of development for these technologies is being pushed faster and harder in the 
commercial world. NASA needs to keep abreast of these new developments, test new 
systems for TOR control, and integrate those that demonstrate their worth into future TOR 
systems. These new technologies will allow NASA astronauts and the general public alike to 
experience SEI missions first-hand through telepresence. 

Powerful new commercial speech synthesis and recognition products are also poised to 
enter the marketplace. NASA should monitor these developments so their capabilities can be 
quickly and cost-effectively integrated into new TOR control interfaces. 

Emerging TOR control technologies and advances in computer simulation may also 
permit development of radically new control interfaces that can greatly increase TOR 
operator productivity and the effective radius of TOR control from thousands to millions of 
kilometers. These new control interfaces, or cyberspace interfaces (CSIs), need to be studied 
by NASA to see how they can best be used to control TORs, and if they lead to new strategies 
for obtaining higher forms of machine autonomy. Many Project Outreach submissions have 
suggested development of CSIs. One submission in particular (#100317) described in broad 
conceptual terms the enormous potential benefits of using these new technologies for TOR 
control. 

IMPLICATIONS OF TOR, CSI, AND TELEPRESENCE CAPABILITIES 

TORs may be used extensively in many phases of SEI operations. A significant 
amount of TOR coordination, mission planning, and real-time retasking will be required, 
especially for complex and TOR-intensive operations like MTV assembly or lunar base 
construction. If CSIs are used for TOR control, even more coordination may be necessary, 
because TOR operators will be sensorially centered at the remote site where their TOR 
operates, rather than at their control stations. 
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By making analogies to certain military operations and practices, it is conjectured that 
TOR command, control, and communications (C3) centers will be required to efficiently and 
safely perform TOR coordination and task planning. Depending upon the sophistication of 
TOR control available in the time frame of SEI, TOR C3 centers may be required at each 
mayor extraterrestrial SEI operations center. On the other hand, if CSIs can effectively 
extend man's control range over TORs and if TORs can eventually be given greater 
autonomy, a single TOR control station located on Earth could direct TOR operations in 
space, on the Moon, and perhaps even in the far term on Mars. Although different 
terminology is used by the author, submission #100337 suggests development of such TOR 
C3 centers. 

The manpower, power, habitat, and communications requirements this suggestion 
implies must be studied by NASA and included in future SEI architecture studies. The most 
significant implication of widely using TORs and incorporating telepresence controls into SEI 
would be the greatly increased communications burden SEI space networks may have to 
support. If one conjectures that HDTV-like display devices are used for stereoscopic control 
of each TOR, then roughly two HDTV channels will have to be supplied for every TOR that is 
controlled from a distant location. New developments in image compression and distributed 
simulation technologies will be required to reduce TOR command and control 
communications requirements and make cyberspace interfaces a reality. NASA should 
carefully monitor developments in these areas. 

SEI ROBOT MISSION PLANNING 

As the SEI program proceeds over the next quarter century, SEI operations will 
increase in scope and complexity. Succeeding generations of SEI robots will depend upon 
and exploit data collected from previous SEI and NASA missions. Data collection 
requirements on early missions should therefore be carefully determined with later SEI 
mission-planning needs in mind. Synergies may exist between early SEI data collection 
efforts and later exploratory, construction, or resource extraction missions. If high -resolution 
data are collected on early exploratory missions, they may prove useful for many purposes 
and could reduce the cost and complexity of follow-on robotic systems, such as lunar rovers or 
base-construction robots. In addition, such data collection efforts will provide scientists and 
prospectors with an unprecedented geologic record of the lunar and perhaps Martian 
surfaces. 

High-resolution imaging (0.1 m) of the Moon is feasible and could perhaps be carried 
out at a number of wavelengths. NASA should examine innovations in new sensor 
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technologies and in small satellite developments (Lightsats) to see if the Lunar Observer or 
Martian Observer spacecraft should be augmented by new lightweight remote sensing 
systems that could not only provide higher resolution optical imagery, but could also image 
deep, permanently dark craters near the lunar poles. 

ROBOT EXPLORER TEAMS 

A number of surface-exploration robots have been proposed in various Project 
Outreach submissions to perform exploration tasks over various types of terrain 
(submissions #100336, #100815, #100337, #101325, #100339, and #100343). They can be 
grouped according to the mobility and navigational concepts they employ. Tradeoff studies 
need to be conducted comparing various mobility and navigation concepts to select which 
could best fulfill SEI mission objectives. In addition, one submission (#100343) proposes that 
robot teams be used to explore the Martian and lunar surface. Such a team may offer more 
terrain flexibility and may be more cost effective than employing identically configured 
multipurpose complex rovers. 

TRANSITION TO AUTONOMOUS ROBOTS 

A key SEI robotics programmatic issue over the next twenty years will be the 
schedule-development risk for semi -autonomous or autonomous robots. Versatile 
autonomous robots capable of operating in unstructured SEI environments (a planetary 
surface or free-flying LEO) will require the following capabilities: goal-directed navigation, 
system control, propulsion, decisionmaking, image recognition, and perhaps voice 
recognition. These capabilities require development of large, error-free software codes. As 
with present AI software, software development risk must be considered to be very high. 

IOC dates for autonomous robots cannot be predicted and may not be achievable without an 
enormous investment in software development infrastructure. Semi-autonomous robot 
developments will likely trail other SEI development schedules. 

Several submissions (#100342, #100345, #100348, #100442, and #100333) recommend 
that NASA should adapt or develop emerging artificial intelligence technologies, autonomous 
navigation software, and new modular robot control and software standards such as 
NASREM and SOSAS in order to develop autonomous robots. While these standards are 
rather general in nature at this time, NASA can certainly profit from examination of the 
systems. 

Although TORs may be easier to develop than autonomous robots, the latter have an 
advantage in that significantly less manpower and communications may be necessary to 
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support their activities. Each TOR will be controlled by a human operator. For construction 
of SSF or other LEO-based space structures, TOR operators could be located on Earth. TOR 
manpower requirements for an Earth-based TOR C3 center will not be a major SEI 
architectural issue, although the associated communications requirements will be significant. 
On the other hand, TORs on the Moon or Mars may have to be directed from local C3 centers 
because of the communication time delays incurred from Earth. The 2.5-second round-trip 
time delay between the Earth and Moon would render present-day TOR feedback control 
loops unstable. With near-term technology, TORs performing dextrous or complex tasks on 
the Moon must be locally controlled. On the other hand, if advanced TOR control interfaces 
and semi-autonomous TORs can be developed, then TOR control could be extended over 
progressively greater distances. 

Tradeoff studies need to be performed by NASA to find the most cost-effective and 
technically feasible SEI robot development plan, and to determine whether TOR or 
autonomous robot research and development should be emphasized. In addition, such 
assessments could also be used to determine which key subsystem technologies must be 
targeted for further development. If current technology trends continue, TORs equipped 
with CSI control interfaces and some autonomous capabilities will prove to be the preferred 
development option. 

REVIEW NASA’S EVALUATION OF A&R EFFORT FOR SPACE STATION FREEDOM 

The United States space program would be impossible without a level of automation 
and robotics that reflects, to some extent, the general state of the art. However, over the past 
twenty years the dominant role of military and NASA agencies in A&R research and 
development has been sharply reduced while the role of commercial industry has increased 
proportionally. A major challenge to NASA is simply maintaining an awareness of A&R 
advances and how these technologies are being used in new ways in the commercial world 
(use of CAD/CAM technologies in the semiconductor industry for the modular design of 
integrated circuits is one example). Implementation of evolving A&R technologies is an 
enormous challenge to the administration. At the direction of Congress, NASA has 
conducted a continual review of the implementation of A&R within the Space Station 
Freedom. A&R implementation efforts have been reviewed approximately every six months 
since 1985 [Refs. 7-15]. We recommend that the Synthesis Group review NASA’s evaluations 
of the Space Station Freedom effort to see how advanced A&R could best be incorporated in 
to SEI. Such a review will reveal the many difficulties, both human and technological, that 
lie ahead, and at the same time the great motivations for pressing ahead. 
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Appendix A 

SUBMISSION HANDLING, EVALUATION METHODOLOGY, AND AUTOMATION AND 
ROBOTICS PANEL CRITERIA FOR EVALUATING SUBMISSIONS 


Submitters were asked to select the appropriate category for their ideas from 
among those listed in Table A.1. The table shows that all categories received a fair 
number of submissions. Of the 1697 submissions received, 149 (less than 9 percent) were 
judged to be incapable of being screened. Another 105 submissions were received after 
the cutoff date of August 31, 1990. 


Table A.1 

Submissions Distributed by Category 


Category 

Screened 

Not Analyzed 

Architecture 

290 

1 

Systems 

52 

0 

Transportation 

350 

0 

Power 

138 

1 

Life support 

156 

2 

Processing 

75 

3 

Structures 

119 

1 

Communications 

45 

1 

Automation 

52 

1 

Information 

21 

1 

Ground support 

28 

0 

Others 

194 

4 

Undetermined 

28 

134 

Total 

1548 

149 

Received after 8/31/90 

105 



A submission was ruled incapable of being screened if it (1) was marked as 
classified or proprietary or (2) contained no supporting information of any kind. A 
submission marked as either proprietary or classified was automatically destroyed by the 
subcontractor. In such cases, the subcontractor noted who destroyed it, the date, and any 
particulars, then informed the submitter of the destruction of the submission and the 
reason for it. 

As shown in Table A.2, the majority of submissions (63 percent) came from 
individuals, with 22 percent coming from for-profit firms and 5 percent from educational 
institutions. The relatively few submissions from educational institutions may have been 


PRECEDING PAGE BLANK NOT FILMED 
uaU?X .ilWENTJONAlLY #LAi»S 



-60- 


a problem of timing, because Project Outreach’s publicity and submission process began 
in the summertime, when most lower-level schools are closed and most universities have 
reduced staffs and enrollments. 


Table A-2 

Sources of Submission 


Source 

Number 

Submissions 

% of Total 

Individuals 

1061 

63 

For-profit firms 

381 

22 

Educational institutions 

89 

5 

Nonprofit organizations 

72 

4 

Other 

46 

3 

Groups of individuals 

48 

3 

Total 

1697 

100 


Nevertheless, Project Outreach generated broad national interest. All of the states 
except Alaska, Arkansas, and Wyoming were represented, as were five foreign 
countries — Argentina, Australia, Canada, Israel, and Scotland. Interestingly, 40 percent 
of the submissions came from three states — California with 26 percent, Texas with 9 
percent, and Florida with 5 percent 

NASA personnel also contributed to Project Outreach: submissions were received 
from the Johnson Space Center, Goddard Space Flight Center, Marshall Space Flight 
Center, Lewis Research Center, Ames Research Center, Jet Propulsion Laboratory, 
Langley Research Center, the Reston Space Station Program Office, and the Stennis 
Space Center. A total of 121 submissions were received from NASA locations. 

SUBMISSION FORMAT 

Submitters were asked for a two-page summary and simple outline of their idea. 
Submitters were also given the option of submitting an additional ten-page backup 
explanation of their idea. Only 22 percent of the total submissions included backups. 
This had implications for the analysis process, which we discuss below. 
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SUBMISSION HANDLING 

Because of time constraints, RAND was obliged to follow an abbreviated six-month 
schedule. Figure A.1 shows the flow of the process we developed and implemented for 
handling the submissions. Our task involved simultaneously processing the submissions, 
developing a methodology, training the panels, and building the software. This time 
frame allowed no margin for error. 



Fig. A.1 — Flow of submission handling 


During our screening and ranking process, we were, in effect, testing the software 
and the methodology, a highly risky process. We are happy to report they both 
performed well. 

SUBMISSION DATABASE 

For each submission, pertinent background information was logged into the 
database, including the unique ID number of the submission, the reviewer, the date, the 
name of the panel performing the review, and the title or subject of the review. To 
remove bias from the process, the panels did not have information concerning the 
submitter’s name or organization. Reviews of the submissions were entered in a text 
field. Each reviewer was required to briefly explain the reasons for scoring a submission 
as he or she did. 
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PANEL RANKING OF SUBMISSIONS 
Primary Ranking Method 

Submissions were ranked initially using a method based on weighted sums of five 
attribute scores. In this case, the attribute weightings were numbers between zero and 
one that summed to one over the five attributes. These weightings represented the 
consensus of each panel concerning the relative importance of the attribute for the 
panel’s particular technology/mission area. 

Table A.3 presents the screening process weights determined by each panel for 
each of five common attributes. Each submission received a composite score, computed 
by summing over all attributes the product of the attribute score (1-5) and its weight. 
Thus, rankings represent the overall score of a submission relative to all the submissions 
within its panel. Rankings by composite score can be sorted within the Fourth 
Dimension database and recomputed using different attribute weights to perform 
sensitivity analysis. 


Table A»3 

Screening Process Weights Determined for Each Panel 


Panel 

Utility 

Feasibility 

Safety 

Innovativeness 

Cost 

Architecture 

0.30 

0.30 

0.15 

0.20 

0.05 

Transportation 

0.30 

0.25 

0.25 

0.05 

0.15 

Power 

0.25 

0.25 

0.25 

0.10 

0.15 

Human Support 

0.40 

0.25 

0.08 

0.25 

0.02 

Structures 

0.30 

0.25 

0.20 

0.10 

0.15 

Robotics 

0.30 

0.25 

0.01 

0.04 

0.20 

Communications 

0.50 

0.25 

0.01 

0.04 

0.20 

Information 

0.29 

0.23 

0.11 

0.20 

0.17 
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Prloritlzed Ranking Method 

To test the robustness of the screening process, each panel also ranked 
submissions using prioritized attribute ranking methods. In these, the most important 
(primary) attribute is selected, and submissions are ranked according to their scores for 
that attribute alone. Submissions with equal scores on the primary attribute are then 
ranked by their score on the next most important, or secondary attribute. The panels 
found that it was rarely necessary to use a third attribute to rank all the submissions by 
this process. The prioritized ranking of a submission can then be compared with its 
general ranking results to determine if there are significant differences. The lack of 
significant differences in the two ranking systems would indicate that the results are 
somewhat robust. 

In addition, a secondary prioritized ranking was created by reversing the order of 
the first two attributes in the primary ranking. Thus, if safety was the most important 
and utility the second most important attribute for a given panel, the order was reversed. 
This provided a further check on robustness. 

Comparison of Methods 

Figure A.2 shows an example comparison of the results of the rankings from the 
Structures panel submissions. The vertical axis represents the primary rank of a 
submission, and the horizontal axis measures its prioritized rank. The intersection 
points of these rankings are shown by small black boxes or squares. The figure contains 
a 45-degree line from the origin out through the total number of submissions. 
Submissions that had the same primary rank and the same prioritized rank would fall 
directly on the 45-degree line. The “best” submission for this panel would be the one 
closest to the origin, because it would be the one that ranked first in the primary ranking 
or first in the prioritized rankings, or first on both. Thus, the closer that each of the 
small black boxes falls to the 45-degree line, the better the congruence of the two ranking 
methods. Figure A.2 shows that the dark blocks representing the top 20 or 25 
submissions are in the lower left-hand corner, indicating good agreement. The 
agreements of the two ranking methods become less congruent as one moves out into the 
lower-ranked submissions, which is to be expected. 
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Fig. A-2 — Flow of Submission Handling 

Table A.4 compares the percentage of common submissions found in the lists of the 
top 20 submissions as created by the three ranking methods just discussed. The left- 
hand column shows the percentage of submissions that appeared on both the primary 
and “primary prioritized" lists; it indicates that the percentage of overlap of the top 20 
submissions on both lists ranged from 75-85 percent. The right-hand column shows the 
commonalties among three lists: the primary rankings, the “primary prioritized" 
rankings, and the “secondary prioritized” rankings discussed above. This comparison 
was made as a more stringent test of robustness; it also reveals a fairly high correlation 
among the three ranking methods. 

This correlation gives confidence in the consistency of the evaluation method used 
to screen submissions. It shows that whether we extracted the top 20 submissions using 
the prioritized or the primary methods, they would still be nearly the same. 
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Table A-4 

Comparison of Ranking of Top 20 Submissions for Each Panel 


Panel 

Percentage of Submissions Appearing on 
2 lists a 3 lists** 

Architecture 

75 

40 

Transportation 

75 

35 

Power 

85 

75 

Life Support 

80 

55 

Structures 

85 

80 

Communications 

85 

55 

Robotics 

85 

55 

Information 

80 

80 


a Primary and prioritized, 
b Primary, prioritized, and reverse prioritized. 


CRITERIA USED BY AUTOMATION AND ROBOTICS PANEL IN EVALUATING 
SUBMISSIONS 

Utility 

The dimensions considered were performance, efficiency, ease of implementation, 
graceful degradation, complexity, and flexibility/adaptability. 

A score of 

(1) indicates low utility 
(3) indicates moderate utility 
(5) indicates high utility 

Feasibility/Risk 

The contextual dimensions of feasibility we considered included availability of 
devices, availability of techniques, availability of theory, time scale, and level of 
confidence. 
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A score of 

(1) indicates low feasibility/high risk of failure 

(3) indicates moderate feasibility moderate/risk of failure 

(5) indicates high feasibility/low risk of failure 


Safety 

Safety in this case pertains to human safety. Its dimensions include the direct 
consequence of a failure, the system consequences of a failure, and fail-soft/fail-safe 
issues. 

A score of 

(1) indicates an unsafe concept 

(3) indicates a moderately safe concept 

(5) indicates a very safe concept 


Relative Cost 

Cost was considered within the dimensions of development cost, production cost, 
operation cost, and life-cycle cost. 

A score of 

(1) indicates a relatively high-cost concept 
(3) indicates a medium-cost concept 
(5) indicates a low-cost concept 

Innovation 

Innovation was considered within the dimensions of concept, application, and 
implementation. 

A score of 

(1) indicates the concept was not innovative or was innovative but did not bear 
upon SEI problems. 

(2) indicates the concept was novel but not more useful than known solutions. 

(3) indicates the concept was considered to be significantly better than known 
solutions to a given problem. 

(5) indicates the concept provided a solution to a heretofore unknown problem. 
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Appendix B 

LIST OF ALL AUTOMATION AND ROBOTICS SUBMISSIONS 


Table B.l 

List of Robotics Submissions 


Submission ID 

Title/Subject 

100333 

Three Dimensional Reactive Navigation 

100334 

Attempt to Introduce Human Life to Mars 

100335 

Homeostatic Control for Robot Survivability 

100336 

Computer Simulated Teleoperation 

100337 

The Robot Workshop 

100338 

EVA Equivalent Space Telemanipulation System 

100339 

Automated Subsurface Sampling by Coring Penetration 

100340 

Surface -to- Orbit Collection System 

100341 

Telerobotics in SEI Surface Operations 

100342 

Self-Adaptive, Sealable Real Time Control Architecture for Robot Vehicles 

100343 

Lewis and Clark Expedition II 

100344 

Surface Resource Extruder (SRE) 

100345 

Advanced Control Architecture to Support Various Missions, Robot 
Applications 

100346 

World Model 

100347 

Aluminum Coated Composite Robot Arms with Embedded Fiber Optic Sensors 

100348 

Use of Next Generation Control Techniques for Robot/Machine Control 
Systems 

100349 

Robotic Space (Walker) System 

100375 

Multi -Function Control Boards 

100376 

Sun Rover 

100377 

Solar-Powered Cricket 

100378 

Robotic Assembly of Large Lunar Structures 

100442 

Artificial Intelligence Systems for Space Applications 
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Submission ID 
100593 

100603 

100644 

100695 

100788 

100815 

100827 

100970 

100971 

101067 

101293 

101317 

101318 

101319 

101320 

101321 

101322 

101323 

101324 

101325 

101439 

101440 

101469 

101514 

101536 

101537 

101635 

101668 


Title/Subject 

The Mass Distribution Construction System Automation, Robotics, and 
Teleoperator 

The Mass Distribution Construction System Ground Support, Simulation and 
Testing 

Orbital Assembly and Maintenance 

Space Robotics: A Highly Dexterous Robot with Adaptable Control Strategies 

Space Based Nondestructive Evaluation 

Wheeled Articulating Rover Propulsion 

Telepresence and Commercial Mission Objectives 

Five-Parameter Characterization of Robots 

Robotics: Waystation Carousels 

Image Processing by the Lunar Rovers 

Autonomous Free Flying Robots for 0-G Space 

Creation of a Virtual Environment for Teleoperation 

Integrated Kinematics, Dynamics, and Artificial Intelligence Robotic 

Development 

The Automation of Modular Structural Assemblies 
Small Space Dog Robot 
Competition for Design of Exploration Robots 
Roboman — A Man-Like Robot 

Repair Robot and Rover Vehicle for Robotic Maintenance 
Smart Components 
Crawling Rover/Manipulator Project 
Bayesian Control Systems 

Task Allocation Among Humans, Teleoperated Devices and Robots 
The Moon-Mars Autonomous Resource Management System 
Remote Tug Vehicle (RTV) 

Superiority of Supervised Robotics 
Magnetoencephalography for Reduced-Delay Control 
Robot Precursors to Planetary Surfaces 
Mars Exploration by Interactive Telepresence 
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Submission ID 

Title/Subject 

200881 

401569 

On-Orbit Assembly, Servicing, and/or Maintenance Using Systems 
Exploration of Mars and the Moon 

NOTE: There are 52 Robotics submissions. 
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